2023-10-30T12:28:59.020 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-10-30T12:28:59.021 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --owner scheduled_leonidus@teuthology --name leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032 --verbose -- /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/orig.config.yaml 2023-10-30T12:28:59.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T12:28:59.154 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032 branch: wip-lusov-balance-after-stop description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7441032' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-lusov-balance-after-stop ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: false mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-balance-after-stop sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 owner: scheduled_leonidus@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 8397 sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 sleep_before_teardown: 0 subset: 464/1024 suite: fs suite_branch: wip-lusov-balance-after-stop suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi008.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiIcfgwrGC9+v3RXe8vNJeksoOyZsreQ1gqopuMNh7A6b/pQ2xJyT5FrS1lYTxOOiBa46uoctB44j3nvKR85mw= smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrOaSl/1vsdItzZTYKMwfRP6iubhYHTiH89lRxHjBRZoyL5v6NbgZ2wXiui+zGvgcvVlPNuVTJ+jWYb+TFCaoA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-10-30_06:34:16 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-30T12:28:59.154 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa; will attempt to use it 2023-10-30T12:28:59.155 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks 2023-10-30T12:28:59.155 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-30T12:28:59.157 INFO:teuthology.task.internal:Checking packages... 2023-10-30T12:28:59.178 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '00c1c13cea21a686b046584596fbeac5315a7d71' 2023-10-30T12:28:59.178 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-30T12:28:59.179 INFO:teuthology.packaging:ref: None 2023-10-30T12:28:59.179 INFO:teuthology.packaging:tag: None 2023-10-30T12:28:59.179 INFO:teuthology.packaging:branch: wip-lusov-balance-after-stop 2023-10-30T12:28:59.180 INFO:teuthology.packaging:sha1: 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:28:59.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-lusov-balance-after-stop 2023-10-30T12:28:59.408 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:28:59.408 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-30T12:28:59.420 INFO:teuthology.task.internal:no buildpackages task found 2023-10-30T12:28:59.420 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-30T12:28:59.435 INFO:teuthology.task.internal:Saving configuration 2023-10-30T12:28:59.453 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-30T12:28:59.463 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:28:59.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:23:14.879626', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHCjdoYg1vsbsGqUY0lBibMiEjCIMZu05ELzUjtnI0w/'} 2023-10-30T12:28:59.511 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:23:14.881088', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB9hoNM6dhjcB607jbG94H3aZJj1lm0FUvk7+a/16hCG'} 2023-10-30T12:28:59.511 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-30T12:28:59.523 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-10-30T12:28:59.523 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-10-30T12:28:59.523 INFO:teuthology.run_tasks:Running task console_log... 2023-10-30T12:28:59.604 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83a6892c10>, signals=[15]) 2023-10-30T12:28:59.605 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-30T12:28:59.616 INFO:teuthology.task.internal:Opening connections... 2023-10-30T12:28:59.616 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:28:59.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:28:59.693 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:28:59.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:28:59.771 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-30T12:28:59.778 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2023-10-30T12:28:59.784 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2023-10-30T12:28:59.784 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2023-10-30T12:28:59.830 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-30T12:28:59.830 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Ubuntu" 2023-10-30T12:28:59.831 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="22.04" 2023-10-30T12:28:59.831 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-30T12:28:59.831 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_CODENAME=jammy 2023-10-30T12:28:59.831 INFO:teuthology.orchestra.run.smithi008.stdout:ID=ubuntu 2023-10-30T12:28:59.831 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE=debian 2023-10-30T12:28:59.832 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-30T12:28:59.832 INFO:teuthology.orchestra.run.smithi008.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-30T12:28:59.832 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-30T12:28:59.832 INFO:teuthology.orchestra.run.smithi008.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-30T12:28:59.832 INFO:teuthology.orchestra.run.smithi008.stdout:UBUNTU_CODENAME=jammy 2023-10-30T12:28:59.833 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2023-10-30T12:28:59.853 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2023-10-30T12:28:59.858 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2023-10-30T12:28:59.859 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-10-30T12:28:59.906 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-30T12:28:59.906 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2023-10-30T12:28:59.906 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2023-10-30T12:28:59.906 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-30T12:28:59.907 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2023-10-30T12:28:59.907 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2023-10-30T12:28:59.907 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2023-10-30T12:28:59.907 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-30T12:28:59.907 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-30T12:28:59.908 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-30T12:28:59.908 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-30T12:28:59.908 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2023-10-30T12:28:59.908 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-10-30T12:28:59.936 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-30T12:28:59.948 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-30T12:28:59.960 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-30T12:28:59.961 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2023-10-30T12:28:59.964 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2023-10-30T12:28:59.968 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-30T12:28:59.976 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-30T12:28:59.977 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2023-10-30T12:29:00.012 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2023-10-30T12:29:00.020 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-30T12:29:00.080 INFO:teuthology.run_tasks:Running task kernel... 2023-10-30T12:29:00.101 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-30T12:29:00.102 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-30T12:29:00.102 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-30T12:29:00.103 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T12:29:00.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:29:00.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-10-30T12:29:00.273 DEBUG:teuthology.task.kernel:sha1 for client.0 is eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:29:00.273 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "eacf00cc51e204f401b4d5d4f84825a68f373c74"... 2023-10-30T12:29:00.273 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-10-30T12:29:00.279 INFO:teuthology.orchestra.run.smithi008.stdout:5.15.0-56-generic 2023-10-30T12:29:00.279 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:29:00.280 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-30T12:29:00.280 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-10-30T12:29:00.281 DEBUG:teuthology.orchestra.run.smithi008:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-30T12:29:00.333 INFO:teuthology.orchestra.run.smithi008.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-30T12:29:00.460 INFO:teuthology.orchestra.run.smithi008.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-30T12:29:00.460 INFO:teuthology.orchestra.run.smithi008.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-30T12:29:00.460 INFO:teuthology.orchestra.run.smithi008.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-30T12:29:00.461 INFO:teuthology.orchestra.run.smithi008.stderr:hint: 2023-10-30T12:29:00.461 INFO:teuthology.orchestra.run.smithi008.stderr:hint: git config --global init.defaultBranch 2023-10-30T12:29:00.461 INFO:teuthology.orchestra.run.smithi008.stderr:hint: 2023-10-30T12:29:00.461 INFO:teuthology.orchestra.run.smithi008.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-30T12:29:00.461 INFO:teuthology.orchestra.run.smithi008.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-30T12:29:00.462 INFO:teuthology.orchestra.run.smithi008.stderr:hint: 2023-10-30T12:29:00.462 INFO:teuthology.orchestra.run.smithi008.stderr:hint: git branch -m 2023-10-30T12:29:00.462 INFO:teuthology.orchestra.run.smithi008.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-30T12:29:00.464 DEBUG:teuthology.orchestra.run.smithi008:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-30T12:29:00.533 DEBUG:teuthology.orchestra.run.smithi008:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-30T12:29:00.587 DEBUG:teuthology.orchestra.run.smithi008:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-30T12:30:02.784 INFO:teuthology.orchestra.run.smithi008.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-30T12:30:02.784 INFO:teuthology.orchestra.run.smithi008.stderr: * [new branch] main -> origin/main 2023-10-30T12:30:02.816 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190312 -> 20190312 2023-10-30T12:30:02.816 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190416 -> 20190416 2023-10-30T12:30:02.816 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190514 -> 20190514 2023-10-30T12:30:02.817 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190618 -> 20190618 2023-10-30T12:30:02.817 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190717 -> 20190717 2023-10-30T12:30:02.817 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190815 -> 20190815 2023-10-30T12:30:02.817 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20190923 -> 20190923 2023-10-30T12:30:02.817 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20191022 -> 20191022 2023-10-30T12:30:02.818 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20191215 -> 20191215 2023-10-30T12:30:02.818 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200122 -> 20200122 2023-10-30T12:30:02.821 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200316 -> 20200316 2023-10-30T12:30:02.821 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200421 -> 20200421 2023-10-30T12:30:02.821 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200519 -> 20200519 2023-10-30T12:30:02.822 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200619 -> 20200619 2023-10-30T12:30:02.822 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200721 -> 20200721 2023-10-30T12:30:02.822 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200817 -> 20200817 2023-10-30T12:30:02.822 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20200918 -> 20200918 2023-10-30T12:30:02.822 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20201022 -> 20201022 2023-10-30T12:30:02.823 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20201118 -> 20201118 2023-10-30T12:30:02.823 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20201218 -> 20201218 2023-10-30T12:30:02.823 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210208 -> 20210208 2023-10-30T12:30:02.823 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210315 -> 20210315 2023-10-30T12:30:02.823 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210511 -> 20210511 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210716 -> 20210716 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210818 -> 20210818 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20210919 -> 20210919 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20211027 -> 20211027 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20211216 -> 20211216 2023-10-30T12:30:02.824 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220209 -> 20220209 2023-10-30T12:30:02.825 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220310 -> 20220310 2023-10-30T12:30:02.825 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220411 -> 20220411 2023-10-30T12:30:02.825 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220509 -> 20220509 2023-10-30T12:30:02.825 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220610 -> 20220610 2023-10-30T12:30:02.825 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220708 -> 20220708 2023-10-30T12:30:02.826 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220815 -> 20220815 2023-10-30T12:30:02.826 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20220913 -> 20220913 2023-10-30T12:30:02.826 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20221012 -> 20221012 2023-10-30T12:30:02.826 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20221109 -> 20221109 2023-10-30T12:30:02.826 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20221214 -> 20221214 2023-10-30T12:30:02.827 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230117 -> 20230117 2023-10-30T12:30:02.827 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230210 -> 20230210 2023-10-30T12:30:02.827 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230310 -> 20230310 2023-10-30T12:30:02.828 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230404 -> 20230404 2023-10-30T12:30:02.828 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230515 -> 20230515 2023-10-30T12:30:02.828 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230625 -> 20230625 2023-10-30T12:30:02.829 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230804 -> 20230804 2023-10-30T12:30:02.829 INFO:teuthology.orchestra.run.smithi008.stderr: * [new tag] 20230919 -> 20230919 2023-10-30T12:30:06.719 INFO:teuthology.orchestra.run.smithi008.stderr:Updating files: 49% (1573/3168) Updating files: 50% (1584/3168) Updating files: 51% (1616/3168) Updating files: 52% (1648/3168) Updating files: 53% (1680/3168) Updating files: 54% (1711/3168) Updating files: 55% (1743/3168) Updating files: 56% (1775/3168) Updating files: 57% (1806/3168) Updating files: 58% (1838/3168) Updating files: 59% (1870/3168) Updating files: 60% (1901/3168) Updating files: 61% (1933/3168) Updating files: 62% (1965/3168) Updating files: 63% (1996/3168) Updating files: 64% (2028/3168) Updating files: 64% (2042/3168) Updating files: 65% (2060/3168) Updating files: 66% (2091/3168) Updating files: 67% (2123/3168) Updating files: 68% (2155/3168) Updating files: 69% (2186/3168) Updating files: 70% (2218/3168) Updating files: 71% (2250/3168) Updating files: 72% (2281/3168) Updating files: 73% (2313/3168) Updating files: 74% (2345/3168) Updating files: 75% (2376/3168) Updating files: 76% (2408/3168) Updating files: 77% (2440/3168) Updating files: 78% (2472/3168) Updating files: 79% (2503/3168) Updating files: 80% (2535/3168) Updating files: 81% (2567/3168) Updating files: 81% (2582/3168) Updating files: 82% (2598/3168) Updating files: 83% (2630/3168) Updating files: 84% (2662/3168) Updating files: 85% (2693/3168) Updating files: 86% (2725/3168) Updating files: 87% (2757/3168) Updating files: 88% (2788/3168) Updating files: 89% (2820/3168) Updating files: 90% (2852/3168) Updating files: 91% (2883/3168) Updating files: 92% (2915/3168) Updating files: 93% (2947/3168) Updating files: 94% (2978/3168) Updating files: 95% (3010/3168) Updating files: 96% (3042/3168) Updating files: 97% (3073/3168) Updating files: 98% (3105/3168) Updating files: 99% (3137/3168) Updating files: 100% (3168/3168) Updating files: 100% (3168/3168), done. 2023-10-30T12:30:06.724 INFO:teuthology.orchestra.run.smithi008.stdout:HEAD is now at 2b304bfe Merge branch 'main' into 'main' 2023-10-30T12:30:06.745 INFO:teuthology.task.kernel:Downloading kernel eacf00cc51e204f401b4d5d4f84825a68f373c74 on client.0... 2023-10-30T12:30:06.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=eacf00cc51e204f401b4d5d4f84825a68f373c74 2023-10-30T12:30:06.894 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:30:06.895 DEBUG:teuthology.orchestra.run.smithi008:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-geacf00cc51e2_6.6.0-rc5-geacf00cc51e2-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/ --input-file=- 2023-10-30T12:30:06.897 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi008.front.sepia.ceph.com to complete... 2023-10-30T12:30:10.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30 12:30:09 URL:https://4.chacra.ceph.com/r/kernel/testing/eacf00cc51e204f401b4d5d4f84825a68f373c74/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-geacf00cc51e2/linux-image-6.6.0-rc5-geacf00cc51e2_6.6.0-rc5-geacf00cc51e2-1_amd64.deb [38909658/38909658] -> "/tmp/linux-image.deb" [1] 2023-10-30T12:30:10.035 INFO:teuthology.orchestra.run.smithi008.stderr:FINISHED --2023-10-30 12:30:09-- 2023-10-30T12:30:10.035 INFO:teuthology.orchestra.run.smithi008.stderr:Total wall clock time: 3.1s 2023-10-30T12:30:10.035 INFO:teuthology.orchestra.run.smithi008.stderr:Downloaded: 1 files, 37M in 2.8s (13.5 MB/s) 2023-10-30T12:30:10.038 INFO:teuthology.task.kernel:Installing kernel eacf00cc51e204f401b4d5d4f84825a68f373c74 on client.0... 2023-10-30T12:30:10.038 DEBUG:teuthology.orchestra.run.smithi008:> sudo dpkg -i /tmp/linux-image.deb 2023-10-30T12:30:10.247 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-geacf00cc51e2. 2023-10-30T12:30:12.274 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-30T12:30:12.278 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-30T12:30:12.385 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-image-6.6.0-rc5-geacf00cc51e2 (6.6.0-rc5-geacf00cc51e2-1) ... 2023-10-30T12:30:15.639 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-image-6.6.0-rc5-geacf00cc51e2 (6.6.0-rc5-geacf00cc51e2-1) ... 2023-10-30T12:30:15.769 INFO:teuthology.orchestra.run.smithi008.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub' 2023-10-30T12:30:34.807 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-30T12:30:34.842 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2023-10-30T12:30:35.125 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:35.420 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-30T12:30:35.431 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-30T12:30:35.497 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-30T12:30:35.502 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-30T12:30:35.653 INFO:teuthology.orchestra.run.smithi008.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-30T12:30:35.654 INFO:teuthology.orchestra.run.smithi008.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-30T12:30:35.654 INFO:teuthology.orchestra.run.smithi008.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-30T12:30:35.691 INFO:teuthology.orchestra.run.smithi008.stderr:done 2023-10-30T12:30:35.862 DEBUG:teuthology.orchestra.run.smithi008:> dpkg-deb -c /tmp/linux-image.deb 2023-10-30T12:30:35.870 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./ 2023-10-30T12:30:35.870 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./boot/ 2023-10-30T12:30:35.871 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 4415097 2023-10-30 03:46 ./boot/System.map-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:35.881 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 169233 2023-10-30 03:46 ./boot/config-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:35.881 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11352928 2023-10-30 03:46 ./boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:35.899 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/ 2023-10-30T12:30:35.899 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/ 2023-10-30T12:30:35.899 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/postinst.d/ 2023-10-30T12:30:35.900 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/postrm.d/ 2023-10-30T12:30:35.900 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/preinst.d/ 2023-10-30T12:30:35.900 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./etc/kernel/prerm.d/ 2023-10-30T12:30:35.900 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/ 2023-10-30T12:30:35.900 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/ 2023-10-30T12:30:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:30:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/ 2023-10-30T12:30:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/ 2023-10-30T12:30:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/ 2023-10-30T12:30:35.901 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/ 2023-10-30T12:30:35.902 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 490233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-30T12:30:35.905 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-30T12:30:35.905 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-30T12:30:35.905 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-30T12:30:35.905 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 93113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-30T12:30:35.905 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 106729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-30T12:30:35.906 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 152241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-30T12:30:35.906 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-30T12:30:35.906 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-30T12:30:35.906 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 83889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-30T12:30:35.906 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-30T12:30:35.907 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-30T12:30:35.907 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-30T12:30:35.907 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-30T12:30:35.907 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-30T12:30:35.907 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-30T12:30:35.908 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-30T12:30:35.908 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-30T12:30:35.908 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-30T12:30:35.908 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-30T12:30:35.908 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/ 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/ 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/mce/ 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/cpuid.ko 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kernel/msr.ko 2023-10-30T12:30:35.909 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/ 2023-10-30T12:30:35.910 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 353761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-30T12:30:35.910 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 881713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-30T12:30:35.910 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 2276817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/arch/x86/kvm/kvm.ko 2023-10-30T12:30:35.964 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ 2023-10-30T12:30:35.964 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/af_alg.ko 2023-10-30T12:30:35.964 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/algif_hash.ko 2023-10-30T12:30:35.965 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/algif_skcipher.ko 2023-10-30T12:30:35.965 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ansi_cprng.ko 2023-10-30T12:30:35.965 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/anubis.ko 2023-10-30T12:30:35.965 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/arc4.ko 2023-10-30T12:30:35.966 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/asymmetric_keys/ 2023-10-30T12:30:35.966 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-30T12:30:35.966 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/ 2023-10-30T12:30:35.966 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_memcpy.ko 2023-10-30T12:30:35.966 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_pq.ko 2023-10-30T12:30:35.967 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-30T12:30:35.967 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_tx.ko 2023-10-30T12:30:35.967 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/async_xor.ko 2023-10-30T12:30:35.967 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/async_tx/raid6test.ko 2023-10-30T12:30:35.967 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/authenc.ko 2023-10-30T12:30:35.968 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/authencesn.ko 2023-10-30T12:30:35.968 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blake2b_generic.ko 2023-10-30T12:30:35.968 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blowfish_common.ko 2023-10-30T12:30:35.968 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/blowfish_generic.ko 2023-10-30T12:30:35.968 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/camellia_generic.ko 2023-10-30T12:30:35.969 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast5_generic.ko 2023-10-30T12:30:35.969 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast6_generic.ko 2023-10-30T12:30:35.969 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cast_common.ko 2023-10-30T12:30:35.969 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/ccm.ko 2023-10-30T12:30:35.969 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cmac.ko 2023-10-30T12:30:35.970 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crc32_generic.ko 2023-10-30T12:30:35.970 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cryptd.ko 2023-10-30T12:30:35.970 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_engine.ko 2023-10-30T12:30:35.970 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_simd.ko 2023-10-30T12:30:35.970 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/crypto_user.ko 2023-10-30T12:30:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/cts.ko 2023-10-30T12:30:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/des_generic.ko 2023-10-30T12:30:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/dh_generic.ko 2023-10-30T12:30:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/echainiv.ko 2023-10-30T12:30:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/essiv.ko 2023-10-30T12:30:35.972 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/fcrypt.ko 2023-10-30T12:30:35.972 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/khazad.ko 2023-10-30T12:30:35.972 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lrw.ko 2023-10-30T12:30:35.972 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lz4.ko 2023-10-30T12:30:35.972 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/lz4hc.ko 2023-10-30T12:30:35.973 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/md4.ko 2023-10-30T12:30:35.973 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/michael_mic.ko 2023-10-30T12:30:35.973 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/pcbc.ko 2023-10-30T12:30:35.973 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/pcrypt.ko 2023-10-30T12:30:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/rmd160.ko 2023-10-30T12:30:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/seed.ko 2023-10-30T12:30:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/serpent_generic.ko 2023-10-30T12:30:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 149193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/tcrypt.ko 2023-10-30T12:30:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/tea.ko 2023-10-30T12:30:35.975 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/twofish_common.ko 2023-10-30T12:30:35.975 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/twofish_generic.ko 2023-10-30T12:30:35.975 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/vmac.ko 2023-10-30T12:30:35.975 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/wp512.ko 2023-10-30T12:30:35.975 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xcbc.ko 2023-10-30T12:30:35.976 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xor.ko 2023-10-30T12:30:35.976 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xts.ko 2023-10-30T12:30:35.976 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/crypto/xxhash_generic.ko 2023-10-30T12:30:35.976 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ 2023-10-30T12:30:35.976 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/ 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_extlog.ko 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/acpi_pad.ko 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/apei/ 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/apei/einj.ko 2023-10-30T12:30:35.977 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/ec_sys.ko 2023-10-30T12:30:35.978 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/platform_profile.ko 2023-10-30T12:30:35.978 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/sbs.ko 2023-10-30T12:30:35.978 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/sbshc.ko 2023-10-30T12:30:35.978 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 103025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/acpi/video.ko 2023-10-30T12:30:35.979 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ 2023-10-30T12:30:35.979 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/acard-ahci.ko 2023-10-30T12:30:35.979 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 88865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ahci.ko 2023-10-30T12:30:35.979 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/ahci_platform.ko 2023-10-30T12:30:35.979 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 88289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/libahci.ko 2023-10-30T12:30:35.980 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/libahci_platform.ko 2023-10-30T12:30:35.980 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_acpi.ko 2023-10-30T12:30:35.980 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ali.ko 2023-10-30T12:30:35.980 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_amd.ko 2023-10-30T12:30:35.980 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_artop.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_atiixp.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_atp867x.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cmd640.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cmd64x.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_cypress.ko 2023-10-30T12:30:35.981 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_efar.ko 2023-10-30T12:30:35.982 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt366.ko 2023-10-30T12:30:35.982 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt37x.ko 2023-10-30T12:30:35.982 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-30T12:30:35.982 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-30T12:30:35.982 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_it8213.ko 2023-10-30T12:30:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_it821x.ko 2023-10-30T12:30:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_jmicron.ko 2023-10-30T12:30:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_legacy.ko 2023-10-30T12:30:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_marvell.ko 2023-10-30T12:30:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_mpiix.ko 2023-10-30T12:30:35.984 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_netcell.ko 2023-10-30T12:30:35.984 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ninja32.ko 2023-10-30T12:30:35.984 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ns87410.ko 2023-10-30T12:30:35.984 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_ns87415.ko 2023-10-30T12:30:35.984 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_oldpiix.ko 2023-10-30T12:30:35.985 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_opti.ko 2023-10-30T12:30:35.985 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_optidma.ko 2023-10-30T12:30:35.985 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pcmcia.ko 2023-10-30T12:30:35.985 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-30T12:30:35.985 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-30T12:30:35.986 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_piccolo.ko 2023-10-30T12:30:35.986 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_radisys.ko 2023-10-30T12:30:35.986 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_rdc.ko 2023-10-30T12:30:35.986 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_rz1000.ko 2023-10-30T12:30:35.986 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sch.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_serverworks.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sil680.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_sl82c105.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_triflex.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pata_via.ko 2023-10-30T12:30:35.987 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/pdc_adma.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_inic162x.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 83345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_mv.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_nv.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_promise.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_qstor.ko 2023-10-30T12:30:35.988 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sil.ko 2023-10-30T12:30:35.989 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sil24.ko 2023-10-30T12:30:35.989 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sis.ko 2023-10-30T12:30:35.989 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_svw.ko 2023-10-30T12:30:35.989 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_sx4.ko 2023-10-30T12:30:35.989 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_uli.ko 2023-10-30T12:30:35.990 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_via.ko 2023-10-30T12:30:35.990 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ata/sata_vsc.ko 2023-10-30T12:30:35.990 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/auxdisplay/ 2023-10-30T12:30:35.990 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/auxdisplay/ks0108.ko 2023-10-30T12:30:35.990 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/bcma/ 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 124473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/bcma/bcma.ko 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/ 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/aoe/ 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 113665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/aoe/aoe.ko 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/drbd/ 2023-10-30T12:30:35.991 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 965585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/drbd/drbd.ko 2023-10-30T12:30:35.992 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 173225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/floppy.ko 2023-10-30T12:30:35.992 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/mtip32xx/ 2023-10-30T12:30:35.992 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 94297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-30T12:30:35.992 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 115793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/nbd.ko 2023-10-30T12:30:35.992 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/null_blk/ 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 110449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/null_blk/null_blk.ko 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 98193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/pktcdvd.ko 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 263769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/rbd.ko 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/xen-blkback/ 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 98145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-30T12:30:35.993 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/zram/ 2023-10-30T12:30:35.994 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/block/zram/zram.ko 2023-10-30T12:30:35.994 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ 2023-10-30T12:30:35.994 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/applicom.ko 2023-10-30T12:30:35.994 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hangcheck-timer.ko 2023-10-30T12:30:35.995 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/ 2023-10-30T12:30:35.995 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-30T12:30:35.995 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-30T12:30:35.995 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-30T12:30:35.995 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/via-rng.ko 2023-10-30T12:30:35.996 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-30T12:30:35.996 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ 2023-10-30T12:30:35.996 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-30T12:30:35.996 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 148601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-30T12:30:35.996 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-30T12:30:35.997 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 139313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-30T12:30:35.997 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-30T12:30:35.997 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-30T12:30:35.997 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/lp.ko 2023-10-30T12:30:35.997 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/mwave/ 2023-10-30T12:30:35.998 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 84097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/mwave/mwave.ko 2023-10-30T12:30:35.998 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/nvram.ko 2023-10-30T12:30:35.998 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/ppdev.ko 2023-10-30T12:30:35.998 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tlclk.ko 2023-10-30T12:30:35.998 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/ 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-30T12:30:35.999 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-30T12:30:36.000 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-30T12:30:36.000 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/ 2023-10-30T12:30:36.000 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-30T12:30:36.000 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-30T12:30:36.000 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/ 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-palmas.ko 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-si5351.ko 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-twl6040.ko 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/clk/clk-wm831x.ko 2023-10-30T12:30:36.001 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/ 2023-10-30T12:30:36.002 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-30T12:30:36.002 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-30T12:30:36.002 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-30T12:30:36.002 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ 2023-10-30T12:30:36.002 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 243025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/ccp/ccp.ko 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/ 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/ 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-30T12:30:36.003 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 438761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-30T12:30:36.004 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-30T12:30:36.004 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-30T12:30:36.004 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/padlock-aes.ko 2023-10-30T12:30:36.004 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/padlock-sha.ko 2023-10-30T12:30:36.004 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/virtio/ 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 68857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dca/ 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dca/dca.ko 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/dw/ 2023-10-30T12:30:36.005 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-30T12:30:36.006 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ioat/ 2023-10-30T12:30:36.006 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 122569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-30T12:30:36.006 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/ 2023-10-30T12:30:36.006 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 79953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/amd64_edac.ko 2023-10-30T12:30:36.006 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/e752x_edac.ko 2023-10-30T12:30:36.007 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 63617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/edac_mce_amd.ko 2023-10-30T12:30:36.007 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i3000_edac.ko 2023-10-30T12:30:36.007 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i3200_edac.ko 2023-10-30T12:30:36.007 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i5100_edac.ko 2023-10-30T12:30:36.007 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i5400_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i7300_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i7core_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/i82975x_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/ie31200_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 56825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/sb_edac.ko 2023-10-30T12:30:36.008 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/edac/x38_edac.ko 2023-10-30T12:30:36.009 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/ 2023-10-30T12:30:36.009 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-gpio.ko 2023-10-30T12:30:36.009 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max14577.ko 2023-10-30T12:30:36.009 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max77693.ko 2023-10-30T12:30:36.009 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-max8997.ko 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-palmas.ko 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/ 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-30T12:30:36.010 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/efi/ 2023-10-30T12:30:36.011 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-30T12:30:36.011 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-30T12:30:36.011 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/ 2023-10-30T12:30:36.011 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-30T12:30:36.011 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-arizona.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-da9052.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-da9055.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-dln2.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 77849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-30T12:30:36.012 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-generic.ko 2023-10-30T12:30:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-ich.ko 2023-10-30T12:30:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-30T12:30:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-kempld.ko 2023-10-30T12:30:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-30T12:30:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max7300.ko 2023-10-30T12:30:36.014 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max730x.ko 2023-10-30T12:30:36.014 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-max732x.ko 2023-10-30T12:30:36.014 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-30T12:30:36.014 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-30T12:30:36.014 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-30T12:30:36.015 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-30T12:30:36.015 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-sch.ko 2023-10-30T12:30:36.015 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-30T12:30:36.015 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-30T12:30:36.015 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-vx855.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-30T12:30:36.016 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-30T12:30:36.017 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/ 2023-10-30T12:30:36.017 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-a4tech.ko 2023-10-30T12:30:36.017 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-apple.ko 2023-10-30T12:30:36.017 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-appleir.ko 2023-10-30T12:30:36.017 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-aureal.ko 2023-10-30T12:30:36.018 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-axff.ko 2023-10-30T12:30:36.018 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-belkin.ko 2023-10-30T12:30:36.018 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cherry.ko 2023-10-30T12:30:36.018 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-chicony.ko 2023-10-30T12:30:36.018 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cp2112.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-cypress.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-dr.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-elecom.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-elo.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-emsff.ko 2023-10-30T12:30:36.019 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ezkey.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gaff.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-generic.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gt683r.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-gyration.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-30T12:30:36.020 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-holtekff.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-hyperv.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-icade.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-kensington.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-keytouch.ko 2023-10-30T12:30:36.021 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-kye.ko 2023-10-30T12:30:36.022 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lcpower.ko 2023-10-30T12:30:36.022 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-led.ko 2023-10-30T12:30:36.022 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lenovo.ko 2023-10-30T12:30:36.022 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-lg-g15.ko 2023-10-30T12:30:36.022 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 80993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-logitech.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-magicmouse.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-microsoft.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-monterey.ko 2023-10-30T12:30:36.023 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-multitouch.ko 2023-10-30T12:30:36.024 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ntrig.ko 2023-10-30T12:30:36.024 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-ortek.ko 2023-10-30T12:30:36.024 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-penmount.ko 2023-10-30T12:30:36.024 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-petalynx.ko 2023-10-30T12:30:36.024 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-picolcd.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-pl.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-plantronics.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-primax.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-rmi.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-30T12:30:36.025 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-common.ko 2023-10-30T12:30:36.026 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-30T12:30:36.026 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-30T12:30:36.026 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-30T12:30:36.026 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-30T12:30:36.026 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-roccat.ko 2023-10-30T12:30:36.027 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-saitek.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-samsung.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sjoy.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sony.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-speedlink.ko 2023-10-30T12:30:36.028 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-steelseries.ko 2023-10-30T12:30:36.029 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-sunplus.ko 2023-10-30T12:30:36.029 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-30T12:30:36.029 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-tivo.ko 2023-10-30T12:30:36.029 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-tmff.ko 2023-10-30T12:30:36.029 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-topseed.ko 2023-10-30T12:30:36.030 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-twinhan.ko 2023-10-30T12:30:36.030 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 71073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-uclogic.ko 2023-10-30T12:30:36.030 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-waltop.ko 2023-10-30T12:30:36.030 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 143473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-wiimote.ko 2023-10-30T12:30:36.030 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-xinmo.ko 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-zpff.ko 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid-zydacron.ko 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 267825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/hid.ko 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/uhid.ko 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/ 2023-10-30T12:30:36.031 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 217873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hid/wacom.ko 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/ 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 64201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_balloon.ko 2023-10-30T12:30:36.032 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_utils.ko 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 305465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hv/hv_vmbus.ko 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/abituguru.ko 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/abituguru3.ko 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-30T12:30:36.033 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ad7414.ko 2023-10-30T12:30:36.034 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ad7418.ko 2023-10-30T12:30:36.034 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adc128d818.ko 2023-10-30T12:30:36.034 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1025.ko 2023-10-30T12:30:36.034 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 97977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1026.ko 2023-10-30T12:30:36.034 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1029.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm1031.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adm9240.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ads7828.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7410.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7411.ko 2023-10-30T12:30:36.035 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 74433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7462.ko 2023-10-30T12:30:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7470.ko 2023-10-30T12:30:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7475.ko 2023-10-30T12:30:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/adt7x10.ko 2023-10-30T12:30:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/amc6821.ko 2023-10-30T12:30:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/applesmc.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asb100.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asc7621.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/atxp1.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/coretemp.ko 2023-10-30T12:30:36.037 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-30T12:30:36.038 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-30T12:30:36.038 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-30T12:30:36.038 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 99601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/dme1737.ko 2023-10-30T12:30:36.038 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ds1621.ko 2023-10-30T12:30:36.038 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ds620.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc1403.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc2103.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/emc6w201.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f71805f.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 94457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f71882fg.ko 2023-10-30T12:30:36.039 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/f75375s.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/fam15h_power.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/fschmd.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/g760a.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/g762.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/gl518sm.ko 2023-10-30T12:30:36.040 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/gl520sm.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/hih6130.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/i5500_temp.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/i5k_amb.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ibmaem.ko 2023-10-30T12:30:36.041 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ibmpex.ko 2023-10-30T12:30:36.042 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ina209.ko 2023-10-30T12:30:36.042 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ina2xx.ko 2023-10-30T12:30:36.042 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 130449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/it87.ko 2023-10-30T12:30:36.042 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/jc42.ko 2023-10-30T12:30:36.042 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/k10temp.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/k8temp.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lineage-pem.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm63.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm73.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm75.ko 2023-10-30T12:30:36.043 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm77.ko 2023-10-30T12:30:36.044 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm78.ko 2023-10-30T12:30:36.044 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm80.ko 2023-10-30T12:30:36.044 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm83.ko 2023-10-30T12:30:36.044 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm85.ko 2023-10-30T12:30:36.044 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm87.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm90.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm92.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 108841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm93.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95234.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95241.ko 2023-10-30T12:30:36.045 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/lm95245.ko 2023-10-30T12:30:36.046 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc2945.ko 2023-10-30T12:30:36.046 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4151.ko 2023-10-30T12:30:36.046 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4215.ko 2023-10-30T12:30:36.046 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4222.ko 2023-10-30T12:30:36.046 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4245.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4260.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/ltc4261.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max16065.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max1619.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max1668.ko 2023-10-30T12:30:36.047 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max197.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6639.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6650.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/max6697.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/mcp3021.ko 2023-10-30T12:30:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6683.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 119817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6775-core.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct6775.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/nct7802.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 82337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pc87360.ko 2023-10-30T12:30:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pc87427.ko 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pcf8591.ko 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-30T12:30:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-30T12:30:36.051 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-30T12:30:36.051 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-30T12:30:36.051 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-30T12:30:36.051 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 82577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-30T12:30:36.051 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/powr1220.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch5627.ko 2023-10-30T12:30:36.052 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch5636.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sht15.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sht21.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/shtc1.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/sis5595.ko 2023-10-30T12:30:36.053 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47b397.ko 2023-10-30T12:30:36.054 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47m1.ko 2023-10-30T12:30:36.054 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/smsc47m192.ko 2023-10-30T12:30:36.054 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/thmc50.ko 2023-10-30T12:30:36.054 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp102.ko 2023-10-30T12:30:36.054 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp103.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp401.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/tmp421.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/via-cputemp.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/via686a.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/vt1211.ko 2023-10-30T12:30:36.055 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/vt8231.ko 2023-10-30T12:30:36.056 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83627ehf.ko 2023-10-30T12:30:36.056 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83627hf.ko 2023-10-30T12:30:36.056 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 71345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83781d.ko 2023-10-30T12:30:36.056 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 63289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83791d.ko 2023-10-30T12:30:36.056 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 71321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83792d.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83793.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 119505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83795.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83l785ts.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/w83l786ng.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-30T12:30:36.057 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-30T12:30:36.058 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/ 2023-10-30T12:30:36.058 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/ 2023-10-30T12:30:36.058 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-30T12:30:36.058 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-30T12:30:36.058 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/ 2023-10-30T12:30:36.059 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-30T12:30:36.061 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-30T12:30:36.061 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-30T12:30:36.062 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-30T12:30:36.062 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-30T12:30:36.062 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-30T12:30:36.062 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-30T12:30:36.062 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-30T12:30:36.063 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-30T12:30:36.063 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-30T12:30:36.063 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-30T12:30:36.063 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-30T12:30:36.063 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-30T12:30:36.064 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-30T12:30:36.064 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-30T12:30:36.064 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-30T12:30:36.064 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-30T12:30:36.064 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-30T12:30:36.065 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-30T12:30:36.066 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-30T12:30:36.066 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-30T12:30:36.066 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-30T12:30:36.066 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-30T12:30:36.067 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-mux.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-smbus.ko 2023-10-30T12:30:36.068 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/i2c-stub.ko 2023-10-30T12:30:36.069 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/ 2023-10-30T12:30:36.069 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-30T12:30:36.069 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-30T12:30:36.069 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-30T12:30:36.069 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 273609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 879969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_core.ko 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 69337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 324217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-30T12:30:36.070 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 97985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 282273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 493073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-30T12:30:36.071 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx4/ 2023-10-30T12:30:36.072 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 419737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-30T12:30:36.072 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx5/ 2023-10-30T12:30:36.072 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 739025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-30T12:30:36.072 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mthca/ 2023-10-30T12:30:36.072 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 296625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 153697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/usnic/ 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 168969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ 2023-10-30T12:30:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-30T12:30:36.074 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 263081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-30T12:30:36.074 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/iser/ 2023-10-30T12:30:36.074 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 100241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-30T12:30:36.074 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/isert/ 2023-10-30T12:30:36.074 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 111425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srp/ 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 128753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srpt/ 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 129529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/ 2023-10-30T12:30:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/evbug.ko 2023-10-30T12:30:36.076 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/ff-memless.ko 2023-10-30T12:30:36.076 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/ 2023-10-30T12:30:36.076 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-30T12:30:36.076 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-30T12:30:36.076 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/gameport/gameport.ko 2023-10-30T12:30:36.077 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/joydev.ko 2023-10-30T12:30:36.077 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/ 2023-10-30T12:30:36.077 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-30T12:30:36.077 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-30T12:30:36.077 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lm8323.ko 2023-10-30T12:30:36.078 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/lm8333.ko 2023-10-30T12:30:36.079 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-30T12:30:36.079 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-30T12:30:36.079 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-30T12:30:36.079 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-30T12:30:36.079 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-30T12:30:36.080 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-30T12:30:36.080 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/qt1070.ko 2023-10-30T12:30:36.080 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/qt2160.ko 2023-10-30T12:30:36.080 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-30T12:30:36.080 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/stowaway.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-30T12:30:36.081 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/matrix-keymap.ko 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/ 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/appletouch.ko 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/bcm5974.ko 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 123681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/cyapatp.ko 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-30T12:30:36.082 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-30T12:30:36.083 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 335977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/psmouse.ko 2023-10-30T12:30:36.083 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/sermouse.ko 2023-10-30T12:30:36.083 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-30T12:30:36.083 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-30T12:30:36.083 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/rmi4/ 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 109873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/altera_ps2.ko 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/arc_ps2.ko 2023-10-30T12:30:36.084 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ct82c710.ko 2023-10-30T12:30:36.085 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-30T12:30:36.085 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/parkbd.ko 2023-10-30T12:30:36.085 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/pcips2.ko 2023-10-30T12:30:36.085 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/ps2mult.ko 2023-10-30T12:30:36.085 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/serio_raw.ko 2023-10-30T12:30:36.086 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/serio/serport.ko 2023-10-30T12:30:36.086 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/input/sparse-keymap.ko 2023-10-30T12:30:36.086 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/ 2023-10-30T12:30:36.086 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/amd/ 2023-10-30T12:30:36.086 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-30T12:30:36.087 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/ 2023-10-30T12:30:36.087 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-88pm860x.ko 2023-10-30T12:30:36.087 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-adp5520.ko 2023-10-30T12:30:36.087 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-bd2802.ko 2023-10-30T12:30:36.087 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-blinkm.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-da903x.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-da9052.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-gpio.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3530.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3533.ko 2023-10-30T12:30:36.088 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm355x.ko 2023-10-30T12:30:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lm3642.ko 2023-10-30T12:30:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lp3944.ko 2023-10-30T12:30:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-lp8788.ko 2023-10-30T12:30:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-max8997.ko 2023-10-30T12:30:36.089 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-mc13783.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca9532.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca955x.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-pca963x.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-ss4200.ko 2023-10-30T12:30:36.090 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-tca6507.ko 2023-10-30T12:30:36.091 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-30T12:30:36.091 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/leds-wm8350.ko 2023-10-30T12:30:36.091 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ 2023-10-30T12:30:36.091 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-30T12:30:36.091 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-30T12:30:36.092 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-30T12:30:36.093 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/ 2023-10-30T12:30:36.093 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/bcache/ 2023-10-30T12:30:36.093 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 654969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/bcache/bcache.ko 2023-10-30T12:30:36.093 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-bio-prison.ko 2023-10-30T12:30:36.093 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-bufio.ko 2023-10-30T12:30:36.094 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-cache-smq.ko 2023-10-30T12:30:36.094 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 156009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-cache.ko 2023-10-30T12:30:36.094 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 106897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-crypt.ko 2023-10-30T12:30:36.094 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-delay.ko 2023-10-30T12:30:36.094 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-era.ko 2023-10-30T12:30:36.095 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-flakey.ko 2023-10-30T12:30:36.095 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-log-userspace.ko 2023-10-30T12:30:36.095 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-log.ko 2023-10-30T12:30:36.095 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-mirror.ko 2023-10-30T12:30:36.095 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-multipath.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-queue-length.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-raid.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-region-hash.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-round-robin.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-service-time.ko 2023-10-30T12:30:36.096 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 109593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-snapshot.ko 2023-10-30T12:30:36.097 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-switch.ko 2023-10-30T12:30:36.097 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 181865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-thin-pool.ko 2023-10-30T12:30:36.097 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-verity.ko 2023-10-30T12:30:36.097 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/dm-zero.ko 2023-10-30T12:30:36.097 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/faulty.ko 2023-10-30T12:30:36.098 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/linear.ko 2023-10-30T12:30:36.098 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/multipath.ko 2023-10-30T12:30:36.098 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/persistent-data/ 2023-10-30T12:30:36.098 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 199937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-30T12:30:36.098 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid0.ko 2023-10-30T12:30:36.099 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 113073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid1.ko 2023-10-30T12:30:36.099 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 158681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid10.ko 2023-10-30T12:30:36.099 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 395721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/md/raid456.ko 2023-10-30T12:30:36.220 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/ 2023-10-30T12:30:36.221 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/ 2023-10-30T12:30:36.221 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 178137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptbase.ko 2023-10-30T12:30:36.221 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 63385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptctl.ko 2023-10-30T12:30:36.221 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptfc.ko 2023-10-30T12:30:36.221 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptlan.ko 2023-10-30T12:30:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 123321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptsas.ko 2023-10-30T12:30:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 79041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptscsih.ko 2023-10-30T12:30:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/message/fusion/mptspi.ko 2023-10-30T12:30:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/ 2023-10-30T12:30:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm800.ko 2023-10-30T12:30:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm805.ko 2023-10-30T12:30:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/88pm80x.ko 2023-10-30T12:30:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/arizona-i2c.ko 2023-10-30T12:30:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 123721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/arizona.ko 2023-10-30T12:30:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/bcm590xx.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/dln2.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/janz-cmodio.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/kempld-core.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lm3533-core.ko 2023-10-30T12:30:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-30T12:30:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lp3943.ko 2023-10-30T12:30:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lpc_ich.ko 2023-10-30T12:30:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/lpc_sch.ko 2023-10-30T12:30:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/max8907.ko 2023-10-30T12:30:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-30T12:30:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-30T12:30:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/menf21bmc.ko 2023-10-30T12:30:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-30T12:30:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-30T12:30:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/pcf50633.ko 2023-10-30T12:30:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-30T12:30:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/retu-mfd.ko 2023-10-30T12:30:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 80425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/si476x-core.ko 2023-10-30T12:30:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/sm501.ko 2023-10-30T12:30:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/tps65010.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/tps6507x.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/viperboard.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/vx855.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/wl1273-core.ko 2023-10-30T12:30:36.228 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/mfd/wm8994.ko 2023-10-30T12:30:36.229 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ 2023-10-30T12:30:36.229 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-30T12:30:36.229 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ad525x_dpot.ko 2023-10-30T12:30:36.229 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/altera-stapl/ 2023-10-30T12:30:36.229 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-30T12:30:36.230 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/apds9802als.ko 2023-10-30T12:30:36.230 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/apds990x.ko 2023-10-30T12:30:36.230 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/bh1770glc.ko 2023-10-30T12:30:36.230 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/ 2023-10-30T12:30:36.230 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-30T12:30:36.231 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/c2port/core.ko 2023-10-30T12:30:36.231 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/cb710/ 2023-10-30T12:30:36.231 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/cb710/cb710.ko 2023-10-30T12:30:36.231 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ds1682.ko 2023-10-30T12:30:36.231 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/dummy-irq.ko 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/echo/ 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/echo/echo.ko 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/ 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/at24.ko 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-30T12:30:36.232 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/eeprom/max6875.ko 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/enclosure.ko 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/genwqe/ 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 152201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/hmc6352.ko 2023-10-30T12:30:36.233 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/hpilo.ko 2023-10-30T12:30:36.234 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ibmasm/ 2023-10-30T12:30:36.234 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-30T12:30:36.234 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ics932s401.ko 2023-10-30T12:30:36.234 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/isl29003.ko 2023-10-30T12:30:36.235 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/isl29020.ko 2023-10-30T12:30:36.235 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/ 2023-10-30T12:30:36.235 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-30T12:30:36.235 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-30T12:30:36.235 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/ 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei-me.ko 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 56521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei-txe.ko 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 310113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/mei/mei.ko 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/phantom.ko 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ti-st/ 2023-10-30T12:30:36.236 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 74265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tifm_7xx1.ko 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tifm_core.ko 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/tsl2550.ko 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_balloon.ko 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_vmci/ 2023-10-30T12:30:36.237 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 183609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-30T12:30:36.238 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ 2023-10-30T12:30:36.238 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/bonding/ 2023-10-30T12:30:36.238 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 490129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/bonding/bonding.ko 2023-10-30T12:30:36.238 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/ 2023-10-30T12:30:36.238 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/b53/ 2023-10-30T12:30:36.239 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 106033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-30T12:30:36.239 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-30T12:30:36.239 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-30T12:30:36.239 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-30T12:30:36.239 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 453737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/dummy.ko 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/eql.ko 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/ 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-30T12:30:36.240 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-30T12:30:36.241 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 130553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-30T12:30:36.241 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 68849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-30T12:30:36.241 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/ 2023-10-30T12:30:36.241 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-30T12:30:36.241 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-30T12:30:36.242 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-30T12:30:36.242 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 99113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-30T12:30:36.242 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/adaptec/ 2023-10-30T12:30:36.242 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-30T12:30:36.242 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/agere/ 2023-10-30T12:30:36.243 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 69201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-30T12:30:36.243 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/alteon/ 2023-10-30T12:30:36.243 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-30T12:30:36.243 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/altera/ 2023-10-30T12:30:36.243 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/ 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 87297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/ 2023-10-30T12:30:36.244 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-30T12:30:36.245 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 97337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-30T12:30:36.245 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-30T12:30:36.245 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 104673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-30T12:30:36.245 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 90257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 81569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 69729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/ 2023-10-30T12:30:36.246 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-30T12:30:36.247 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 176761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-30T12:30:36.247 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-30T12:30:36.247 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1421497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-30T12:30:36.247 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 128569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-30T12:30:36.247 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 132089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 312081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/ 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 323513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-30T12:30:36.248 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/ 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 193497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 347417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-30T12:30:36.249 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1044433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-30T12:30:36.250 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-30T12:30:36.250 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 166193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-30T12:30:36.250 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-30T12:30:36.250 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-30T12:30:36.250 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/ 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 204345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/ 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 69449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-30T12:30:36.251 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-30T12:30:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 141225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-30T12:30:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-30T12:30:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-30T12:30:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-30T12:30:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/ 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/dnet.ko 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/ 2023-10-30T12:30:36.253 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-30T12:30:36.254 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 281137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-30T12:30:36.254 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ethoc.ko 2023-10-30T12:30:36.254 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fealnx.ko 2023-10-30T12:30:36.254 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fujitsu/ 2023-10-30T12:30:36.254 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-30T12:30:36.255 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ 2023-10-30T12:30:36.255 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 90465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-30T12:30:36.255 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-30T12:30:36.255 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 295577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-30T12:30:36.255 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 584497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 246209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 925697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-30T12:30:36.256 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-30T12:30:36.257 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 302849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-30T12:30:36.257 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igb/ 2023-10-30T12:30:36.257 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 514681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-30T12:30:36.257 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-30T12:30:36.257 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 93593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-30T12:30:36.258 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-30T12:30:36.258 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 734305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-30T12:30:36.258 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-30T12:30:36.258 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 151769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-30T12:30:36.258 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 86921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/jme.ko 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/ 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 102289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 122649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/ 2023-10-30T12:30:36.259 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-30T12:30:36.260 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 727601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-30T12:30:36.260 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 278937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-30T12:30:36.260 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-30T12:30:36.260 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-30T12:30:36.260 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1113993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 95665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-30T12:30:36.261 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/ 2023-10-30T12:30:36.262 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-30T12:30:36.262 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 108649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-30T12:30:36.262 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/ 2023-10-30T12:30:36.262 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-30T12:30:36.262 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-30T12:30:36.263 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/neterion/ 2023-10-30T12:30:36.263 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 150905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-30T12:30:36.263 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/nvidia/ 2023-10-30T12:30:36.263 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 136721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-30T12:30:36.263 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/ 2023-10-30T12:30:36.264 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-30T12:30:36.264 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-30T12:30:36.266 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/ 2023-10-30T12:30:36.266 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-30T12:30:36.266 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 227897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-30T12:30:36.266 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 87081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-30T12:30:36.266 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 672049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/rdc/ 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/ 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-30T12:30:36.267 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 83985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 189801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/ 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 104241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-30T12:30:36.268 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sfc/ 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1053921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/silan/ 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/ 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-30T12:30:36.269 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 63337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-30T12:30:36.270 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/ 2023-10-30T12:30:36.270 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-30T12:30:36.270 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-30T12:30:36.270 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-30T12:30:36.270 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 68177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/ 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-30T12:30:36.271 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 486961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/ 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 103337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 166161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-30T12:30:36.272 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/tehuti/ 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ti/ 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 86529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/ 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 80273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-30T12:30:36.273 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 83633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-30T12:30:36.274 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/ 2023-10-30T12:30:36.274 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-30T12:30:36.274 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-30T12:30:36.274 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/xircom/ 2023-10-30T12:30:36.274 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-30T12:30:36.275 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/ 2023-10-30T12:30:36.275 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/defxx.ko 2023-10-30T12:30:36.275 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/skfp/ 2023-10-30T12:30:36.275 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 216985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-30T12:30:36.275 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 77345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/geneve.ko 2023-10-30T12:30:36.276 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/hyperv/ 2023-10-30T12:30:36.276 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 235521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-30T12:30:36.276 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ieee802154/ 2023-10-30T12:30:36.276 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-30T12:30:36.276 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ifb.ko 2023-10-30T12:30:36.277 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ipvlan/ 2023-10-30T12:30:36.277 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 83769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-30T12:30:36.277 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/macvlan.ko 2023-10-30T12:30:36.277 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/macvtap.ko 2023-10-30T12:30:36.277 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/ 2023-10-30T12:30:36.278 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-30T12:30:36.278 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-30T12:30:36.278 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-30T12:30:36.278 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-30T12:30:36.278 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mdio.ko 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/mii.ko 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/netconsole.ko 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/nlmon.ko 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/ 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-30T12:30:36.279 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-30T12:30:36.280 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/ 2023-10-30T12:30:36.280 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/amd.ko 2023-10-30T12:30:36.280 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-30T12:30:36.280 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-30T12:30:36.280 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/bcm87xx.ko 2023-10-30T12:30:36.281 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/broadcom.ko 2023-10-30T12:30:36.281 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/cicada.ko 2023-10-30T12:30:36.281 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/davicom.ko 2023-10-30T12:30:36.281 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/et1011c.ko 2023-10-30T12:30:36.281 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/icplus.ko 2023-10-30T12:30:36.282 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/lxt.ko 2023-10-30T12:30:36.282 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 82049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/marvell.ko 2023-10-30T12:30:36.282 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 76177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/micrel.ko 2023-10-30T12:30:36.282 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/national.ko 2023-10-30T12:30:36.282 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 97857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/phylink.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/qsemi.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/realtek.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/smsc.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/ste10Xp.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/phy/vitesse.ko 2023-10-30T12:30:36.283 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/plip/ 2023-10-30T12:30:36.284 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/plip/plip.ko 2023-10-30T12:30:36.284 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ 2023-10-30T12:30:36.284 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-30T12:30:36.284 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_async.ko 2023-10-30T12:30:36.284 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-30T12:30:36.285 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-30T12:30:36.285 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-30T12:30:36.285 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pppoe.ko 2023-10-30T12:30:36.285 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pppox.ko 2023-10-30T12:30:36.285 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/ppp/pptp.ko 2023-10-30T12:30:36.286 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/sb1000.ko 2023-10-30T12:30:36.286 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/slip/ 2023-10-30T12:30:36.286 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/slip/slip.ko 2023-10-30T12:30:36.286 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/sungem_phy.ko 2023-10-30T12:30:36.286 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 64385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/tap.ko 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/ 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 106169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team.ko 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-30T12:30:36.287 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_random.ko 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 87017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/veth.ko 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vmxnet3/ 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 144297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vxlan/ 2023-10-30T12:30:36.288 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 297329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/vxlan/vxlan.ko 2023-10-30T12:30:36.289 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/xen-netback/ 2023-10-30T12:30:36.289 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 144649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-30T12:30:36.289 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ntb/ 2023-10-30T12:30:36.289 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ntb/ntb.ko 2023-10-30T12:30:36.289 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/ 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/ 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 264937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/nvme-core.ko 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 102209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/nvme/host/nvme.ko 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/ 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 133841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport.ko 2023-10-30T12:30:36.290 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_cs.ko 2023-10-30T12:30:36.291 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 84465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_pc.ko 2023-10-30T12:30:36.291 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/parport/parport_serial.ko 2023-10-30T12:30:36.291 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/ 2023-10-30T12:30:36.291 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/ 2023-10-30T12:30:36.291 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/pci-stub.ko 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pci/xen-pcifront.ko 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/ 2023-10-30T12:30:36.292 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/i82092.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 145809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/pd6729.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 112889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-30T12:30:36.293 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/ 2023-10-30T12:30:36.294 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/broadcom/ 2023-10-30T12:30:36.294 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-30T12:30:36.294 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/samsung/ 2023-10-30T12:30:36.294 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-30T12:30:36.294 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/ 2023-10-30T12:30:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/intel/ 2023-10-30T12:30:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-30T12:30:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/ 2023-10-30T12:30:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/ 2023-10-30T12:30:36.295 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-30T12:30:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-30T12:30:36.297 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/ 2023-10-30T12:30:36.297 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-30T12:30:36.297 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/acerhdf.ko 2023-10-30T12:30:36.297 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-30T12:30:36.297 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-30T12:30:36.298 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-30T12:30:36.298 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 127569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-30T12:30:36.298 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-30T12:30:36.298 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-30T12:30:36.298 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-30T12:30:36.299 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-30T12:30:36.299 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-30T12:30:36.299 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/hdaps.ko 2023-10-30T12:30:36.299 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-30T12:30:36.299 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/ 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/intel_ips.ko 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-30T12:30:36.300 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-30T12:30:36.301 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-30T12:30:36.301 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-30T12:30:36.301 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-30T12:30:36.301 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-30T12:30:36.301 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-30T12:30:36.302 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-30T12:30:36.302 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/platform/x86/wmi.ko 2023-10-30T12:30:36.302 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/ 2023-10-30T12:30:36.302 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ 2023-10-30T12:30:36.302 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-30T12:30:36.303 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/da9030_battery.ko 2023-10-30T12:30:36.304 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/da9052-battery.ko 2023-10-30T12:30:36.304 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-30T12:30:36.304 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-30T12:30:36.304 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-30T12:30:36.304 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-30T12:30:36.305 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/gpio-charger.ko 2023-10-30T12:30:36.305 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-30T12:30:36.305 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-30T12:30:36.305 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max14577_charger.ko 2023-10-30T12:30:36.305 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max17040_battery.ko 2023-10-30T12:30:36.306 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max17042_battery.ko 2023-10-30T12:30:36.306 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max8903_charger.ko 2023-10-30T12:30:36.306 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/max8925_power.ko 2023-10-30T12:30:36.306 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-30T12:30:36.306 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/sbs-battery.ko 2023-10-30T12:30:36.307 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/test_power.ko 2023-10-30T12:30:36.307 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-30T12:30:36.307 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-30T12:30:36.307 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm831x_power.ko 2023-10-30T12:30:36.307 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/power/supply/wm8350_power.ko 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/ 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/ 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/ 2023-10-30T12:30:36.308 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/clients/pps_parport.ko 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/pps/pps_core.ko 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ptp.ko 2023-10-30T12:30:36.309 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ptp/ptp_kvm.ko 2023-10-30T12:30:36.310 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/ 2023-10-30T12:30:36.310 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-30T12:30:36.310 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-30T12:30:36.310 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-30T12:30:36.310 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9052.ko 2023-10-30T12:30:36.311 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9055.ko 2023-10-30T12:30:36.311 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-da9063.ko 2023-10-30T12:30:36.311 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-30T12:30:36.311 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 68577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-30T12:30:36.311 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-30T12:30:36.312 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-em3027.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-30T12:30:36.313 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max6900.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8907.ko 2023-10-30T12:30:36.314 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8925.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8997.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-max8998.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-palmas.ko 2023-10-30T12:30:36.315 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-30T12:30:36.316 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-30T12:30:36.316 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-30T12:30:36.316 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-30T12:30:36.316 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-30T12:30:36.316 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-30T12:30:36.317 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-30T12:30:36.318 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/rtc/rtc-x1205.ko 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 77617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-9xxx.ko 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-sas.ko 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/3w-xxxx.ko 2023-10-30T12:30:36.319 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 135185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/BusLogic.ko 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/a100u2w.ko 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aacraid/ 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 272305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 100617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/advansys.ko 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/ 2023-10-30T12:30:36.320 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 356265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 295801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic94xx/ 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 189185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/am53c974.ko 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/arcmsr/ 2023-10-30T12:30:36.321 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 106881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-30T12:30:36.322 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/atp870u.ko 2023-10-30T12:30:36.322 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/be2iscsi/ 2023-10-30T12:30:36.322 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 246985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-30T12:30:36.322 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bfa/ 2023-10-30T12:30:36.322 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1171945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bfa/bfa.ko 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2fc/ 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 237273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2i/ 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 115369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ch.ko 2023-10-30T12:30:36.323 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/csiostor/ 2023-10-30T12:30:36.324 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 308921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-30T12:30:36.324 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/ 2023-10-30T12:30:36.324 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-30T12:30:36.324 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 76145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-30T12:30:36.324 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-30T12:30:36.325 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-30T12:30:36.325 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 146689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-30T12:30:36.325 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 75553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/dc395x.ko 2023-10-30T12:30:36.325 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/dmx3191d.ko 2023-10-30T12:30:36.325 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esas2r/ 2023-10-30T12:30:36.326 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 212425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-30T12:30:36.326 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/esp_scsi.ko 2023-10-30T12:30:36.326 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/ 2023-10-30T12:30:36.326 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-30T12:30:36.326 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 158505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fdomain.ko 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fnic/ 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 263489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/fnic/fnic.ko 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 201817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hpsa.ko 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hptiop.ko 2023-10-30T12:30:36.327 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/hv_storvsc.ko 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/imm.ko 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/initio.ko 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 266537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ipr.ko 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ips.ko 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/isci/ 2023-10-30T12:30:36.328 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 317153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/isci/isci.ko 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libfc/ 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 314865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libfc/libfc.ko 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 145585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libiscsi.ko 2023-10-30T12:30:36.329 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-30T12:30:36.330 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libsas/ 2023-10-30T12:30:36.330 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 216665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/libsas/libsas.ko 2023-10-30T12:30:36.330 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/lpfc/ 2023-10-30T12:30:36.330 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 2197249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-30T12:30:36.330 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/ 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 310617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 84721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/megaraid.ko 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mpt3sas/ 2023-10-30T12:30:36.331 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 659777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvsas/ 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 119561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/mvumi.ko 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/ 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 81625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-30T12:30:36.332 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-30T12:30:36.333 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-30T12:30:36.333 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-30T12:30:36.333 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pm8001/ 2023-10-30T12:30:36.333 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 364105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-30T12:30:36.333 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 123617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/pmcraid.ko 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ppa.ko 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla1280.ko 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/ 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1876625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 85289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-30T12:30:36.334 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla4xxx/ 2023-10-30T12:30:36.335 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 559033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-30T12:30:36.335 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/qlogicfas408.ko 2023-10-30T12:30:36.335 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/raid_class.ko 2023-10-30T12:30:36.335 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 238825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_debug.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 173937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 279313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 97729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-30T12:30:36.336 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/ses.ko 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 115889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/st.ko 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/stex.ko 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 162121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/virtio_scsi.ko 2023-10-30T12:30:36.337 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-30T12:30:36.338 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/wd719x.ko 2023-10-30T12:30:36.338 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/scsi/xen-scsifront.ko 2023-10-30T12:30:36.338 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ssb/ 2023-10-30T12:30:36.338 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 129441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ssb/ssb.ko 2023-10-30T12:30:36.338 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/ 2023-10-30T12:30:36.339 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/iscsi/ 2023-10-30T12:30:36.339 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 736601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-30T12:30:36.339 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/loopback/ 2023-10-30T12:30:36.339 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-30T12:30:36.339 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_file.ko 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_iblock.ko 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 930321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_mod.ko 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_pscsi.ko 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 122521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/target_core_user.ko 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/tcm_fc/ 2023-10-30T12:30:36.340 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 75593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/ 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/ 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-30T12:30:36.341 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-30T12:30:36.342 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-30T12:30:36.342 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-30T12:30:36.342 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-30T12:30:36.342 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-30T12:30:36.342 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-30T12:30:36.343 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ipwireless/ 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-30T12:30:36.344 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/moxa.ko 2023-10-30T12:30:36.345 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 50953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/mxser.ko 2023-10-30T12:30:36.345 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 108601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/n_gsm.ko 2023-10-30T12:30:36.345 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/n_hdlc.ko 2023-10-30T12:30:36.345 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/nozomi.ko 2023-10-30T12:30:36.345 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/ 2023-10-30T12:30:36.346 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/ 2023-10-30T12:30:36.346 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-30T12:30:36.346 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-30T12:30:36.346 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-30T12:30:36.346 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/altera_uart.ko 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/arc_uart.ko 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/jsm/ 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 103073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/rp2.ko 2023-10-30T12:30:36.347 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/ 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/core/ 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 531265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-30T12:30:36.348 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/ 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio.ko 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_aec.ko 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_cif.ko 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-30T12:30:36.349 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_mf624.ko 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_netx.ko 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_pci_generic.ko 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/uio/uio_sercos3.ko 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/ 2023-10-30T12:30:36.350 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/c67x00/ 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 144377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-30T12:30:36.351 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/ 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 87089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/cdc-acm.ko 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/usblp.ko 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/class/usbtmc.ko 2023-10-30T12:30:36.352 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/common/ 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/common/ulpi.ko 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc2/ 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/ 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-30T12:30:36.353 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 444553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/ 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/ 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-30T12:30:36.354 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-30T12:30:36.355 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-30T12:30:36.355 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-30T12:30:36.355 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-30T12:30:36.355 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 92489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-30T12:30:36.355 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-30T12:30:36.356 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 122081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-30T12:30:36.356 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-30T12:30:36.356 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-30T12:30:36.356 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-30T12:30:36.356 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 85609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/ 2023-10-30T12:30:36.357 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-30T12:30:36.358 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-30T12:30:36.358 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-30T12:30:36.358 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-30T12:30:36.358 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-30T12:30:36.358 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-30T12:30:36.359 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-30T12:30:36.360 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 62185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-30T12:30:36.360 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-30T12:30:36.360 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 161225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-30T12:30:36.360 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/ 2023-10-30T12:30:36.360 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 157681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 69705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-30T12:30:36.361 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 56033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-30T12:30:36.362 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 76057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-30T12:30:36.362 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 75073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-30T12:30:36.362 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 73169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-30T12:30:36.362 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 65809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-30T12:30:36.362 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 81505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 156073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/ 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 80113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-30T12:30:36.363 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 87353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-30T12:30:36.364 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-30T12:30:36.364 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-30T12:30:36.364 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/sl811_cs.ko 2023-10-30T12:30:36.364 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-30T12:30:36.364 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-30T12:30:36.365 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/ 2023-10-30T12:30:36.365 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/mdc800.ko 2023-10-30T12:30:36.365 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/image/microtek.ko 2023-10-30T12:30:36.365 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ 2023-10-30T12:30:36.365 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/adutux.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/appledisplay.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/cytherm.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ehset.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/emi26.ko 2023-10-30T12:30:36.366 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/emi62.ko 2023-10-30T12:30:36.367 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ezusb.ko 2023-10-30T12:30:36.367 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/idmouse.ko 2023-10-30T12:30:36.367 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/iowarrior.ko 2023-10-30T12:30:36.367 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-30T12:30:36.367 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/ldusb.ko 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/legousbtower.ko 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/lvstest.ko 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/sisusbvga/ 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 64841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-30T12:30:36.368 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usb3503.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usblcd.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/usbtest.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/uss720.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/misc/yurex.ko 2023-10-30T12:30:36.369 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/mon/ 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/mon/usbmon.ko 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/musb/ 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 301545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/ 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-generic.ko 2023-10-30T12:30:36.370 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/aircable.ko 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ark3116.ko 2023-10-30T12:30:36.371 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-30T12:30:36.372 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ch341.ko 2023-10-30T12:30:36.372 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cp210x.ko 2023-10-30T12:30:36.372 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cyberjack.ko 2023-10-30T12:30:36.372 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-30T12:30:36.372 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/empeg.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/f81232.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 175137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 137081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-30T12:30:36.373 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 124833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/io_ti.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ipaq.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ipw.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ir-usb.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 107521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/keyspan.ko 2023-10-30T12:30:36.374 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-30T12:30:36.375 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-30T12:30:36.375 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-30T12:30:36.375 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mct_u232.ko 2023-10-30T12:30:36.375 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/metro-usb.ko 2023-10-30T12:30:36.375 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mos7720.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 78217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mos7840.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 56337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/mxuport.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/navman.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/omninet.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/opticon.ko 2023-10-30T12:30:36.376 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 190633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/option.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/oti6858.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/pl2303.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/qcaux.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/qcserial.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/quatech2.ko 2023-10-30T12:30:36.377 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/safe_serial.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 51625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/sierra.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ssu100.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/symbolserial.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-30T12:30:36.378 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-30T12:30:36.379 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb_debug.ko 2023-10-30T12:30:36.379 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-30T12:30:36.379 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/usbserial.ko 2023-10-30T12:30:36.379 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/visor.ko 2023-10-30T12:30:36.379 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/whiteheat.ko 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 52913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/uas.ko 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-30T12:30:36.380 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-30T12:30:36.381 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-karma.ko 2023-10-30T12:30:36.382 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-30T12:30:36.382 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-30T12:30:36.382 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-30T12:30:36.382 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-30T12:30:36.382 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 179969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/storage/usb-storage.ko 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/ 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 107001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-30T12:30:36.383 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/ 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/ 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 154297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 108737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/vfio.ko 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 77609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-30T12:30:36.384 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/ 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 113873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost.ko 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 58089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_net.ko 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/vhost/vhost_scsi.ko 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/ 2023-10-30T12:30:36.385 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/ 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-30T12:30:36.386 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/apple_bl.ko 2023-10-30T12:30:36.387 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-30T12:30:36.387 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/bd6107.ko 2023-10-30T12:30:36.387 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-30T12:30:36.387 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-30T12:30:36.387 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lcd.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-30T12:30:36.388 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/ 2023-10-30T12:30:36.389 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ 2023-10-30T12:30:36.390 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ds2482.ko 2023-10-30T12:30:36.390 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/ds2490.ko 2023-10-30T12:30:36.390 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-30T12:30:36.390 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-30T12:30:36.390 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/ 2023-10-30T12:30:36.391 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-30T12:30:36.391 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-30T12:30:36.391 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-30T12:30:36.391 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-30T12:30:36.391 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-30T12:30:36.392 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 56425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 102673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/w1/wire.ko 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/acquirewdt.ko 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/advantechwdt.ko 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-30T12:30:36.393 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-30T12:30:36.394 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-30T12:30:36.394 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-30T12:30:36.394 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-30T12:30:36.394 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-30T12:30:36.394 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/dw_wdt.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/hpwdt.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/i6300esb.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-30T12:30:36.395 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ib700wdt.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ibmasr.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/it87_wdt.ko 2023-10-30T12:30:36.396 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/machzwd.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/nv_tco.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-30T12:30:36.397 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22265 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-30T12:30:36.398 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-30T12:30:36.398 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-30T12:30:36.398 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/retu_wdt.ko 2023-10-30T12:30:36.398 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-30T12:30:36.398 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/softdog.ko 2023-10-30T12:30:36.399 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-30T12:30:36.400 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-30T12:30:36.400 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/via_wdt.ko 2023-10-30T12:30:36.400 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-30T12:30:36.400 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-30T12:30:36.400 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wdt_pci.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/watchdog/xen_wdt.ko 2023-10-30T12:30:36.401 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/ 2023-10-30T12:30:36.402 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-evtchn.ko 2023-10-30T12:30:36.402 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-gntalloc.ko 2023-10-30T12:30:36.402 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-gntdev.ko 2023-10-30T12:30:36.402 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-pciback/ 2023-10-30T12:30:36.402 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 176441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-privcmd.ko 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 71249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xen-scsiback.ko 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xenfs/ 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ 2023-10-30T12:30:36.403 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/9p/ 2023-10-30T12:30:36.404 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 166425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/9p/9p.ko 2023-10-30T12:30:36.404 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/adfs/ 2023-10-30T12:30:36.404 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/adfs/adfs.ko 2023-10-30T12:30:36.404 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/affs/ 2023-10-30T12:30:36.404 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 163593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/affs/affs.ko 2023-10-30T12:30:36.405 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/afs/ 2023-10-30T12:30:36.405 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 932817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/afs/kafs.ko 2023-10-30T12:30:36.405 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/autofs/ 2023-10-30T12:30:36.405 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/autofs/autofs4.ko 2023-10-30T12:30:36.405 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/befs/ 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 64545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/befs/befs.ko 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/bfs/ 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/bfs/bfs.ko 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/binfmt_misc.ko 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/btrfs/ 2023-10-30T12:30:36.406 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 3402585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/btrfs/btrfs.ko 2023-10-30T12:30:36.407 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cachefiles/ 2023-10-30T12:30:36.407 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 306073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cachefiles/cachefiles.ko 2023-10-30T12:30:36.407 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ceph/ 2023-10-30T12:30:36.407 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1323945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ceph/ceph.ko 2023-10-30T12:30:36.407 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/coda/ 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 100801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/coda/coda.ko 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/configfs/ 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 108393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/configfs/configfs.ko 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cramfs/ 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 29625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/cramfs/cramfs.ko 2023-10-30T12:30:36.408 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/dlm/ 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 642577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/dlm/dlm.ko 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/efs/ 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/efs/efs.ko 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/f2fs/ 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1505025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/f2fs/f2fs.ko 2023-10-30T12:30:36.409 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fat/ 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fat/msdos.ko 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/freevxfs/ 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/freevxfs/freevxfs.ko 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fscache/ 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 226481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fscache/fscache.ko 2023-10-30T12:30:36.410 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fuse/ 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/fuse/cuse.ko 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/gfs2/ 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1231529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/gfs2/gfs2.ko 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfs/ 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 141209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfs/hfs.ko 2023-10-30T12:30:36.411 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfsplus/ 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 222817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hfsplus/hfsplus.ko 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hpfs/ 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 192329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/hpfs/hpfs.ko 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/isofs/ 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 74065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/isofs/isofs.ko 2023-10-30T12:30:36.412 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/jfs/ 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 419497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/jfs/jfs.ko 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/lockd/ 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 280313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/lockd/lockd.ko 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/minix/ 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/minix/minix.ko 2023-10-30T12:30:36.413 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/netfs/ 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 124649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/netfs/netfs.ko 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/ 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/blocklayout/ 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 81449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/filelayout/ 2023-10-30T12:30:36.414 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 59449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-30T12:30:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/flexfilelayout/ 2023-10-30T12:30:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 117513 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-30T12:30:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1055649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfs.ko 2023-10-30T12:30:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv2.ko 2023-10-30T12:30:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 120177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv3.ko 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 2041689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs/nfsv4.ko 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/ 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/grace.ko 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfs_common/nfs_acl.ko 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfsd/ 2023-10-30T12:30:36.416 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1516809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nfsd/nfsd.ko 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nilfs2/ 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 592833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nilfs2/nilfs2.ko 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/ 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-celtic.ko 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-centeuro.ko 2023-10-30T12:30:36.417 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-croatian.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-cyrillic.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-gaelic.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-greek.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-iceland.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-inuit.ko 2023-10-30T12:30:36.418 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-roman.ko 2023-10-30T12:30:36.419 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-romanian.ko 2023-10-30T12:30:36.419 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/mac-turkish.ko 2023-10-30T12:30:36.419 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_ascii.ko 2023-10-30T12:30:36.419 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1250.ko 2023-10-30T12:30:36.419 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1251.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp1255.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp737.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp775.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp850.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp852.ko 2023-10-30T12:30:36.420 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp855.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp857.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp860.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp861.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp862.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp863.ko 2023-10-30T12:30:36.421 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp864.ko 2023-10-30T12:30:36.422 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp865.ko 2023-10-30T12:30:36.422 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp866.ko 2023-10-30T12:30:36.422 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp869.ko 2023-10-30T12:30:36.422 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp874.ko 2023-10-30T12:30:36.422 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 94137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp932.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 142673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp936.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 165745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp949.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 118121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_cp950.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_euc-jp.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-1.ko 2023-10-30T12:30:36.423 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-13.ko 2023-10-30T12:30:36.424 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-14.ko 2023-10-30T12:30:36.424 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-15.ko 2023-10-30T12:30:36.424 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-2.ko 2023-10-30T12:30:36.424 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-3.ko 2023-10-30T12:30:36.424 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-4.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-5.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-6.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-7.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_iso8859-9.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-r.ko 2023-10-30T12:30:36.425 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-ru.ko 2023-10-30T12:30:36.426 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_koi8-u.ko 2023-10-30T12:30:36.426 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-30T12:30:36.426 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/nls/nls_utf8.ko 2023-10-30T12:30:36.426 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ntfs/ 2023-10-30T12:30:36.426 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 219417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ntfs/ntfs.ko 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/cluster/ 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 244409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlm/ 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 440737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlmfs/ 2023-10-30T12:30:36.427 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 2592433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2.ko 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/omfs/ 2023-10-30T12:30:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/omfs/omfs.ko 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/overlayfs/ 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 352153 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/overlayfs/overlay.ko 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/pstore/ 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/pstore/ramoops.ko 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx4/ 2023-10-30T12:30:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx4/qnx4.ko 2023-10-30T12:30:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx6/ 2023-10-30T12:30:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 46689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/qnx6/qnx6.ko 2023-10-30T12:30:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/ 2023-10-30T12:30:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_tree.ko 2023-10-30T12:30:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_v1.ko 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/quota/quota_v2.ko 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/reiserfs/ 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 532673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/reiserfs/reiserfs.ko 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/romfs/ 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/romfs/romfs.ko 2023-10-30T12:30:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/ 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/client/ 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 2668121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/client/cifs.ko 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/ 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/cifs_arc4.ko 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/smb/common/cifs_md4.ko 2023-10-30T12:30:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/squashfs/ 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 134641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/squashfs/squashfs.ko 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/sysv/ 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 90721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/sysv/sysv.ko 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/udf/ 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 247737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/udf/udf.ko 2023-10-30T12:30:36.433 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ufs/ 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 162497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/ufs/ufs.ko 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/xfs/ 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 4010729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/fs/xfs/xfs.ko 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/ 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/locking/ 2023-10-30T12:30:36.434 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 75489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/locking/locktorture.ko 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/time/ 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/time/test_udelay.ko 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/kernel/torture.ko 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc-itu-t.ko 2023-10-30T12:30:36.435 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc7.ko 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crc8.ko 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/ 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/libarc4.ko 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/crypto/libdes.ko 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/interval_tree_test.ko 2023-10-30T12:30:36.436 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/libcrc32c.ko 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lru_cache.ko 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/ 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/lz4_compress.ko 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/lz4/lz4hc_compress.ko 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/math/ 2023-10-30T12:30:36.437 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/math/cordic.ko 2023-10-30T12:30:36.438 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/memory-notifier-error-inject.ko 2023-10-30T12:30:36.438 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/notifier-error-inject.ko 2023-10-30T12:30:36.438 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/percpu_test.ko 2023-10-30T12:30:36.438 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/pm-notifier-error-inject.ko 2023-10-30T12:30:36.438 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/raid6/ 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 143433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/raid6/raid6_pq.ko 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/rbtree_test.ko 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/reed_solomon/ 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test-kstrtox.ko 2023-10-30T12:30:36.439 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test-string_helpers.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 4760425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_bpf.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_firmware.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_module.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/test_user_copy.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_bm.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_fsm.ko 2023-10-30T12:30:36.440 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/ts_kmp.ko 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/xz/ 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/lib/xz/xz_dec_test.ko 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/mm/ 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/mm/hwpoison-inject.ko 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ 2023-10-30T12:30:36.441 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/ 2023-10-30T12:30:36.442 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/6lowpan.ko 2023-10-30T12:30:36.442 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_dest.ko 2023-10-30T12:30:36.442 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_fragment.ko 2023-10-30T12:30:36.442 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_hop.ko 2023-10-30T12:30:36.442 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 6985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_mobility.ko 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_routing.ko 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/6lowpan/nhc_udp.ko 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/ 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/garp.ko 2023-10-30T12:30:36.443 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/mrp.ko 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/p8022.ko 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/psnap.ko 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/802/stp.ko 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/8021q/ 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 81737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/8021q/8021q.ko 2023-10-30T12:30:36.444 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/ 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 198337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet.ko 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_fd.ko 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26345 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_rdma.ko 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/9p/9pnet_virtio.ko 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/batman-adv/ 2023-10-30T12:30:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 649297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/batman-adv/batman-adv.ko 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/ 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 63809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/br_netfilter.ko 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 777689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/bridge.ko 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-30T12:30:36.446 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-30T12:30:36.447 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13217 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8385 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-30T12:30:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-30T12:30:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-30T12:30:36.450 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 55929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/ebtables.ko 2023-10-30T12:30:36.450 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-30T12:30:36.450 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-30T12:30:36.450 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ceph/ 2023-10-30T12:30:36.450 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1047497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ceph/libceph.ko 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/core/ 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/core/pktgen.ko 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/ 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 194209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp.ko 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_diag.ko 2023-10-30T12:30:36.451 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 57881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_ipv4.ko 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dccp/dccp_ipv6.ko 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/ 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 390473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/dsa_core.ko 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_brcm.ko 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_dsa.ko 2023-10-30T12:30:36.452 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_none.ko 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/dsa/tag_trailer.ko 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/hsr/ 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 99729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/hsr/hsr.ko 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/6lowpan/ 2023-10-30T12:30:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 45233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 254625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ieee802154.ko 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ah4.ko 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/esp4.ko 2023-10-30T12:30:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/fou.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/gre.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/inet_diag.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_gre.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_tunnel.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ip_vti.ko 2023-10-30T12:30:36.455 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ipcomp.ko 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/ipip.ko 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-30T12:30:36.456 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-30T12:30:36.457 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11305 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-30T12:30:36.458 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18225 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-30T12:30:36.459 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-30T12:30:36.460 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-30T12:30:36.460 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-30T12:30:36.460 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-30T12:30:36.460 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_bic.ko 2023-10-30T12:30:36.460 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_dctcp.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_diag.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_highspeed.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_htcp.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_hybla.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_illinois.ko 2023-10-30T12:30:36.461 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_lp.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_scalable.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_vegas.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_veno.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_westwood.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tcp_yeah.ko 2023-10-30T12:30:36.462 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/tunnel4.ko 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/udp_diag.ko 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 41665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/udp_tunnel.ko 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ah6.ko 2023-10-30T12:30:36.463 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/esp6.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/fou6.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 80897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_gre.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 93577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_tunnel.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ip6_vti.ko 2023-10-30T12:30:36.464 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/ipcomp6.ko 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19017 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/mip6.ko 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-30T12:30:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-30T12:30:36.466 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-30T12:30:36.466 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-30T12:30:36.466 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-30T12:30:36.466 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-30T12:30:36.469 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-30T12:30:36.469 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-30T12:30:36.470 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-30T12:30:36.470 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-30T12:30:36.470 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-30T12:30:36.470 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11121 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-30T12:30:36.470 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25945 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-30T12:30:36.471 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9009 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 74073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/sit.ko 2023-10-30T12:30:36.472 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/tunnel6.ko 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/key/ 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 82833 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/key/af_key.ko 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/ 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125281 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_core.ko 2023-10-30T12:30:36.473 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-30T12:30:36.474 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 25953 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_eth.ko 2023-10-30T12:30:36.474 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ip.ko 2023-10-30T12:30:36.474 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36713 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ip6.ko 2023-10-30T12:30:36.474 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_netlink.ko 2023-10-30T12:30:36.474 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 53241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/l2tp/l2tp_ppp.ko 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/lapb/ 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 47977 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/lapb/lapb.ko 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/ 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/llc.ko 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 267777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/llc/llc2.ko 2023-10-30T12:30:36.475 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mac802154/ 2023-10-30T12:30:36.476 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 290417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mac802154/mac802154.ko 2023-10-30T12:30:36.476 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mpls/ 2023-10-30T12:30:36.476 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/mpls/mpls_gso.ko 2023-10-30T12:30:36.476 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ 2023-10-30T12:30:36.476 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 119585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set.ko 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26913 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 100297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-30T12:30:36.477 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 101441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-30T12:30:36.478 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 102817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-30T12:30:36.478 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 103481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-30T12:30:36.478 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 112849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-30T12:30:36.478 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 60329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-30T12:30:36.478 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 109593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 113129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 112585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 111585 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 114465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-30T12:30:36.479 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ 2023-10-30T12:30:36.480 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 411729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-30T12:30:36.480 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-30T12:30:36.480 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-30T12:30:36.480 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-30T12:30:36.480 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7689 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10681 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-30T12:30:36.481 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-30T12:30:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-30T12:30:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-30T12:30:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10993 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-30T12:30:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conncount.ko 2023-10-30T12:30:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 408257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack.ko 2023-10-30T12:30:36.483 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-30T12:30:36.483 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-30T12:30:36.483 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33761 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-30T12:30:36.483 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 118145 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-30T12:30:36.483 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 107297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-30T12:30:36.484 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10113 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_flow_table.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9593 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 37129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_log_syslog.ko 2023-10-30T12:30:36.485 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 95417 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11857 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_irc.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28929 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_sip.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-30T12:30:36.486 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 44065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-30T12:30:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 615313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nf_tables.ko 2023-10-30T12:30:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 42561 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink.ko 2023-10-30T12:30:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-30T12:30:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30569 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-30T12:30:36.487 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 39785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_log.ko 2023-10-30T12:30:36.488 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-30T12:30:36.488 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-30T12:30:36.488 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10457 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_chain_nat.ko 2023-10-30T12:30:36.488 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_compat.ko 2023-10-30T12:30:36.488 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 40721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_ct.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_fib.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14185 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_hash.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_limit.ko 2023-10-30T12:30:36.489 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_log.ko 2023-10-30T12:30:36.490 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_masq.ko 2023-10-30T12:30:36.490 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14329 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_nat.ko 2023-10-30T12:30:36.490 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13673 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_numgen.ko 2023-10-30T12:30:36.490 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11841 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_osf.ko 2023-10-30T12:30:36.490 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_queue.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_redir.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_reject.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_reject_inet.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_socket.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20369 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_synproxy.ko 2023-10-30T12:30:36.491 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/nft_tproxy.ko 2023-10-30T12:30:36.492 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 89313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/x_tables.ko 2023-10-30T12:30:36.492 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_AUDIT.ko 2023-10-30T12:30:36.492 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-30T12:30:36.492 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-30T12:30:36.492 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 18169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_CT.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_DSCP.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_HL.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_HMARK.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-30T12:30:36.493 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13889 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_LED.ko 2023-10-30T12:30:36.494 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_LOG.ko 2023-10-30T12:30:36.494 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11753 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-30T12:30:36.494 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NETMAP.ko 2023-10-30T12:30:36.494 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NFLOG.ko 2023-10-30T12:30:36.494 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-30T12:30:36.495 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_RATEEST.ko 2023-10-30T12:30:36.495 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-30T12:30:36.495 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 14337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_SECMARK.ko 2023-10-30T12:30:36.495 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-30T12:30:36.495 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-30T12:30:36.496 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17361 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TEE.ko 2023-10-30T12:30:36.496 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TPROXY.ko 2023-10-30T12:30:36.496 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_TRACE.ko 2023-10-30T12:30:36.496 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_addrtype.ko 2023-10-30T12:30:36.496 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_bpf.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cgroup.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11209 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cluster.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_comment.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10465 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connbytes.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11081 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connlabel.ko 2023-10-30T12:30:36.497 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connlimit.ko 2023-10-30T12:30:36.498 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_connmark.ko 2023-10-30T12:30:36.498 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_conntrack.ko 2023-10-30T12:30:36.498 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_cpu.ko 2023-10-30T12:30:36.498 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_dccp.ko 2023-10-30T12:30:36.498 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_devgroup.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8801 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_dscp.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ecn.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_esp.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_hashlimit.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_helper.ko 2023-10-30T12:30:36.499 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7865 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_hl.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11921 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ipcomp.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_iprange.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11785 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_ipvs.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16041 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_l2tp.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8001 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_length.ko 2023-10-30T12:30:36.500 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_limit.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7545 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_mac.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_mark.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11825 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_multiport.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13817 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_nat.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_nfacct.ko 2023-10-30T12:30:36.501 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_osf.ko 2023-10-30T12:30:36.502 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_owner.ko 2023-10-30T12:30:36.502 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_physdev.ko 2023-10-30T12:30:36.502 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_pkttype.ko 2023-10-30T12:30:36.502 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11985 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_policy.ko 2023-10-30T12:30:36.502 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_quota.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_rateest.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_realm.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_recent.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_sctp.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_set.ko 2023-10-30T12:30:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 16089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_socket.ko 2023-10-30T12:30:36.504 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9505 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_state.ko 2023-10-30T12:30:36.504 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9473 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_statistic.ko 2023-10-30T12:30:36.504 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8809 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_string.ko 2023-10-30T12:30:36.504 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 8553 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_tcpmss.ko 2023-10-30T12:30:36.504 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15777 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_tcpudp.ko 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11129 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_time.ko 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netfilter/xt_u32.ko 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netlink/ 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13969 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/netlink/netlink_diag.ko 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/nsh/ 2023-10-30T12:30:36.505 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 12425 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/nsh/nsh.ko 2023-10-30T12:30:36.506 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/ 2023-10-30T12:30:36.506 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 380033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/openvswitch.ko 2023-10-30T12:30:36.506 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10321 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-geneve.ko 2023-10-30T12:30:36.506 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-gre.ko 2023-10-30T12:30:36.506 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 11793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/openvswitch/vport-vxlan.ko 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/packet/ 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13057 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/packet/af_packet_diag.ko 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/ 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 110201 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/phonet.ko 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54697 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/phonet/pn_pep.ko 2023-10-30T12:30:36.507 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/ 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 251481 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds.ko 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 164937 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds_rdma.ko 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 67377 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rds/rds_tcp.ko 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rxrpc/ 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 819657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/rxrpc/rxrpc.ko 2023-10-30T12:30:36.508 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/ 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32241 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_csum.ko 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23577 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_gact.ko 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26873 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_ipt.ko 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34097 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_mirred.ko 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_nat.ko 2023-10-30T12:30:36.509 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 43449 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_pedit.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_police.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_simple.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35297 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_skbedit.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36665 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/act_vlan.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26169 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_basic.ko 2023-10-30T12:30:36.510 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33961 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_bpf.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19233 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_cgroup.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31401 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_flow.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31769 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_fw.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 38489 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_route.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 61409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/cls_u32.ko 2023-10-30T12:30:36.511 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7441 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_cmp.ko 2023-10-30T12:30:36.512 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 13409 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_ipset.ko 2023-10-30T12:30:36.512 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33289 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_meta.ko 2023-10-30T12:30:36.512 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_nbyte.ko 2023-10-30T12:30:36.512 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_text.ko 2023-10-30T12:30:36.512 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7249 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/em_u32.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 21649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_choke.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20849 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_codel.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 34633 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_drr.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 33609 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_fq.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31617 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_fq_codel.ko 2023-10-30T12:30:36.513 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 35433 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_gred.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 49601 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_hfsc.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22089 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_hhf.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 66393 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_htb.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26073 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_ingress.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 30273 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_mqprio.ko 2023-10-30T12:30:36.514 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_mqprio_lib.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26065 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_multiq.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 36137 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_netem.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 27497 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_pie.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10625 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_plug.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 24729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_prio.ko 2023-10-30T12:30:36.515 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 48793 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_qfq.ko 2023-10-30T12:30:36.516 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 32905 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_red.ko 2023-10-30T12:30:36.516 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 26881 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_sfb.ko 2023-10-30T12:30:36.516 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 31353 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_sfq.ko 2023-10-30T12:30:36.516 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28897 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_tbf.ko 2023-10-30T12:30:36.516 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 28105 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sched/sch_teql.ko 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/ 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 929745 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/sctp.ko 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 19033 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sctp/sctp_diag.ko 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/ 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/ 2023-10-30T12:30:36.517 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 336529 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-30T12:30:36.518 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 54737 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-30T12:30:36.518 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 1520161 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/sunrpc.ko 2023-10-30T12:30:36.518 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/xprtrdma/ 2023-10-30T12:30:36.518 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 707721 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-30T12:30:36.518 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/ 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10025 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/diag.ko 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 862641 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/tipc/tipc.ko 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/unix/ 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15729 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/unix/unix_diag.ko 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/ 2023-10-30T12:30:36.519 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 85337 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-30T12:30:36.520 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 68049 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-30T12:30:36.520 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 96193 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock.ko 2023-10-30T12:30:36.520 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 10705 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-30T12:30:36.520 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 17521 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-30T12:30:36.520 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/x25/ 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 125177 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/x25/x25.ko 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/ 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 22657 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_algo.ko 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 20649 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 102257 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/net/xfrm/xfrm_user.ko 2023-10-30T12:30:36.521 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/ 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/lib/ 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 15537 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/kernel/virt/lib/irqbypass.ko 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 639863 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.alias 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 623919 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.alias.bin 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 7663 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin 2023-10-30T12:30:36.522 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 23428 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.alias.bin 2023-10-30T12:30:36.523 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 9821 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.bin 2023-10-30T12:30:36.523 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 70004 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.builtin.modinfo 2023-10-30T12:30:36.523 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 146313 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.dep 2023-10-30T12:30:36.523 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 232548 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.dep.bin 2023-10-30T12:30:36.523 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 210 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.devname 2023-10-30T12:30:36.524 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 72606 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.order 2023-10-30T12:30:36.524 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 879 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.softdep 2023-10-30T12:30:36.524 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 254438 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.symbols 2023-10-30T12:30:36.524 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 309363 2023-10-30 03:46 ./lib/modules/6.6.0-rc5-geacf00cc51e2/modules.symbols.bin 2023-10-30T12:30:36.524 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/ 2023-10-30T12:30:36.525 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/ 2023-10-30T12:30:36.525 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/doc/ 2023-10-30T12:30:36.525 INFO:teuthology.orchestra.run.smithi008.stdout:drwxr-xr-x root/root 0 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/ 2023-10-30T12:30:36.525 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 173 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/changelog.Debian.gz 2023-10-30T12:30:36.525 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-r--r-- root/root 691 2023-10-30 03:46 ./usr/share/doc/linux-image-6.6.0-rc5-geacf00cc51e2/copyright 2023-10-30T12:30:36.526 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:36.526 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:36.527 DEBUG:teuthology.orchestra.run.smithi008:> egrep '(submenu|menuentry.*6.6.0-rc5-geacf00cc51e2).*{' /boot/grub/grub.cfg 2023-10-30T12:30:36.533 INFO:teuthology.orchestra.run.smithi008.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:30:36.534 INFO:teuthology.orchestra.run.smithi008.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-geacf00cc51e2-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:30:36.534 INFO:teuthology.orchestra.run.smithi008.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-geacf00cc51e2-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-30T12:30:36.535 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-30T12:30:36.535 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:36.535 DEBUG:teuthology.orchestra.run.smithi008:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-30T12:30:36.576 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi008.front.sepia.ceph.com to complete... 2023-10-30T12:30:36.968 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub' 2023-10-30T12:30:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-30T12:30:37.006 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2023-10-30T12:30:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:37.196 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-geacf00cc51e2 2023-10-30T12:30:37.468 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-30T12:30:37.474 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-30T12:30:37.518 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-30T12:30:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-30T12:30:37.629 INFO:teuthology.orchestra.run.smithi008.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-30T12:30:37.629 INFO:teuthology.orchestra.run.smithi008.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-30T12:30:37.629 INFO:teuthology.orchestra.run.smithi008.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-30T12:30:37.647 INFO:teuthology.orchestra.run.smithi008.stderr:done 2023-10-30T12:31:08.773 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-30T12:31:08.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:31:08.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:31:27.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-10-30T12:31:36.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:31:36.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:31:39.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-10-30T12:31:51.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:31:51.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:32:23.224 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-10-30T12:32:24.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:32:24.162 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-geacf00cc51e2"... 2023-10-30T12:32:24.163 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-10-30T12:32:24.212 INFO:teuthology.orchestra.run.smithi008.stdout:6.6.0-rc5-geacf00cc51e2 2023-10-30T12:32:24.213 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-geacf00cc51e2 vs 6.6.0-rc5-geacf00cc51e2 2023-10-30T12:32:24.213 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-30T12:32:24.213 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-30T12:32:25.215 DEBUG:teuthology.parallel:result is None 2023-10-30T12:32:25.216 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-30T12:32:25.225 INFO:teuthology.task.internal:Creating test directory... 2023-10-30T12:32:25.226 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T12:32:25.230 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T12:32:25.237 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-30T12:32:25.244 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-30T12:32:25.252 INFO:teuthology.task.internal:Creating archive directory... 2023-10-30T12:32:25.253 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T12:32:25.288 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T12:32:25.324 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-30T12:32:25.333 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-30T12:32:25.333 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T12:32:25.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:32:25.362 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T12:32:25.366 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T12:32:25.427 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:32:25.444 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:32:25.546 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:32:25.573 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T12:32:25.576 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-30T12:32:25.588 INFO:teuthology.task.internal:Configuring sudo... 2023-10-30T12:32:25.589 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T12:32:25.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T12:32:25.612 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-30T12:32:25.622 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-30T12:32:25.623 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T12:32:25.660 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T12:32:25.666 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T12:32:25.718 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T12:32:25.774 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:32:25.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T12:32:25.853 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T12:32:25.860 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T12:32:25.906 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:32:25.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T12:32:25.973 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2023-10-30T12:32:25.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2023-10-30T12:32:26.078 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-30T12:32:26.086 INFO:teuthology.task.internal:Starting timer... 2023-10-30T12:32:26.086 INFO:teuthology.run_tasks:Running task pcp... 2023-10-30T12:32:26.122 INFO:teuthology.run_tasks:Running task selinux... 2023-10-30T12:32:26.130 DEBUG:teuthology.task.selinux:Excluding smithi008: OS 'ubuntu' does not support SELinux 2023-10-30T12:32:26.131 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2023-10-30T12:32:26.131 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-30T12:32:26.131 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-30T12:32:26.132 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-30T12:32:26.132 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-30T12:32:26.141 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-30T12:32:26.146 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-30T12:32:26.265 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-30T12:32:26.281 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-30T12:32:26.283 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi008.front.sepia.ceph.com,smithi046.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-30T12:41:14.613 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi008.front.sepia.ceph.com'), Remote(name='ubuntu@smithi046.front.sepia.ceph.com')] 2023-10-30T12:41:14.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:41:14.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:41:14.694 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-10-30T12:41:14.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-30T12:41:14.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-30T12:41:14.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T12:41:14.887 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-10-30T12:41:15.006 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-30T12:41:15.007 INFO:teuthology.run_tasks:Running task clock... 2023-10-30T12:41:15.018 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-30T12:41:15.019 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T12:41:15.020 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:41:15.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T12:41:15.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:41:15.061 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-30T12:41:15.062 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Command line: ntpd -gq 2023-10-30T12:41:15.062 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: ---------------------------------------------------- 2023-10-30T12:41:15.062 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: ntp-4 is maintained by Network Time Foundation, 2023-10-30T12:41:15.062 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-30T12:41:15.063 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: corporation. Support and training for ntp-4 are 2023-10-30T12:41:15.063 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: available at https://www.nwtime.org/support 2023-10-30T12:41:15.063 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: ---------------------------------------------------- 2023-10-30T12:41:15.064 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: proto: precision = 0.044 usec (-24) 2023-10-30T12:41:15.064 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: basedate set to 2022-02-04 2023-10-30T12:41:15.064 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: gps base set to 2022-02-06 (week 2196) 2023-10-30T12:41:15.065 INFO:teuthology.orchestra.run.smithi008.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:41:15.065 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:41:15.065 INFO:teuthology.orchestra.run.smithi008.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:41:15.066 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:41:15.067 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen and drop on 0 v6wildcard [::]:123 2023-10-30T12:41:15.067 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-30T12:41:15.067 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen normally on 2 lo 127.0.0.1:123 2023-10-30T12:41:15.067 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen normally on 3 ens1f0 172.21.15.8:123 2023-10-30T12:41:15.068 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen normally on 4 lo [::1]:123 2023-10-30T12:41:15.068 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25fe%4]:123 2023-10-30T12:41:15.068 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:15 ntpd[27932]: Listening on routing socket on fd #22 for interface updates 2023-10-30T12:41:15.075 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-30T12:41:15.075 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Command line: ntpd -gq 2023-10-30T12:41:15.075 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: ---------------------------------------------------- 2023-10-30T12:41:15.075 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: ntp-4 is maintained by Network Time Foundation, 2023-10-30T12:41:15.076 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-30T12:41:15.076 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: corporation. Support and training for ntp-4 are 2023-10-30T12:41:15.076 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: available at https://www.nwtime.org/support 2023-10-30T12:41:15.076 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: ---------------------------------------------------- 2023-10-30T12:41:15.077 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: proto: precision = 0.050 usec (-24) 2023-10-30T12:41:15.077 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: basedate set to 2022-02-04 2023-10-30T12:41:15.077 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: gps base set to 2022-02-06 (week 2196) 2023-10-30T12:41:15.078 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:41:15.078 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-30T12:41:15.078 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:41:15.079 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-30T12:41:15.079 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen and drop on 0 v6wildcard [::]:123 2023-10-30T12:41:15.080 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-30T12:41:15.080 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen normally on 2 lo 127.0.0.1:123 2023-10-30T12:41:15.080 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen normally on 3 ens1f0 172.21.15.46:123 2023-10-30T12:41:15.080 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen normally on 4 lo [::1]:123 2023-10-30T12:41:15.080 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2023-10-30T12:41:15.081 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:15 ntpd[28537]: Listening on routing socket on fd #22 for interface updates 2023-10-30T12:41:16.062 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:16 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:16.062 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:16 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:16.075 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:16.075 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:16.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:16 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:17 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:17 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:18 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:18.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:18.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:18.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:18.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:18 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:19.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:19.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:19.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:19.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:19.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:19.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:19 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:20.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:20.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:20.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:20.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:20 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:20.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:20.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:20.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:20.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:20 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:21.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:21 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:22.067 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:22 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.076 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:22.079 INFO:teuthology.orchestra.run.smithi046.stdout:30 Oct 12:41:22 ntpd[28537]: ntpd: time slew -0.000276 s 2023-10-30T12:41:22.079 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew -0.000276s 2023-10-30T12:41:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-30T12:41:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:30 Oct 12:41:22 ntpd[28537]: can't open /var/log/ntpstats/loopstats.20231030: Permission denied 2023-10-30T12:41:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:41:22.156 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-10-30T12:41:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:24.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:24.063 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:24.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:24.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:24.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:24.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:24.064 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/rawstats.20231030: Permission denied 2023-10-30T12:41:24.065 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-30T12:41:24.066 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/peerstats.20231030: Permission denied 2023-10-30T12:41:24.066 INFO:teuthology.orchestra.run.smithi008.stdout:30 Oct 12:41:24 ntpd[27932]: ntpd: time slew +0.003333 s 2023-10-30T12:41:24.066 INFO:teuthology.orchestra.run.smithi008.stdout:ntpd: time slew +0.003333s 2023-10-30T12:41:24.067 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-30T12:41:24.067 INFO:teuthology.orchestra.run.smithi008.stderr:30 Oct 12:41:24 ntpd[27932]: can't open /var/log/ntpstats/loopstats.20231030: Permission denied 2023-10-30T12:41:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:41:24.180 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================== 2023-10-30T12:41:24.180 INFO:teuthology.orchestra.run.smithi008.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:24.180 INFO:teuthology.orchestra.run.smithi008.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:24.180 INFO:teuthology.orchestra.run.smithi008.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:24.180 INFO:teuthology.orchestra.run.smithi008.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-30T12:41:24.182 INFO:teuthology.run_tasks:Running task install... 2023-10-30T12:41:24.191 DEBUG:teuthology.task.install:project ceph 2023-10-30T12:41:24.191 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}} 2023-10-30T12:41:24.192 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'} 2023-10-30T12:41:24.192 INFO:teuthology.task.install:Using flavor: default 2023-10-30T12:41:24.197 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-30T12:41:24.198 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-30T12:41:24.198 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-key list | grep Ceph 2023-10-30T12:41:24.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2023-10-30T12:41:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-30T12:41:24.298 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-30T12:41:24.299 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2023-10-30T12:41:24.299 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-30T12:41:24.299 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-30T12:41:24.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:41:24.336 INFO:teuthology.orchestra.run.smithi008.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-30T12:41:24.391 INFO:teuthology.orchestra.run.smithi008.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-30T12:41:24.391 INFO:teuthology.orchestra.run.smithi008.stdout:uid [ unknown] Ceph.com (release key) 2023-10-30T12:41:24.392 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-30T12:41:24.392 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-30T12:41:24.392 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:41:24.444 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default/ 2023-10-30T12:41:24.445 INFO:teuthology.task.install.deb:Package version is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:41:24.616 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default/ 2023-10-30T12:41:24.617 INFO:teuthology.task.install.deb:Package version is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:41:24.763 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:41:24.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-30T12:41:24.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2023-10-30T12:41:24.953 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-30T12:41:24.977 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:41:24.977 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-30T12:41:24.999 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get update 2023-10-30T12:41:25.071 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-30T12:41:25.153 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-30T12:41:25.193 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-30T12:41:25.193 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-30T12:41:25.202 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-30T12:41:25.221 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-30T12:41:25.231 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy InRelease 2023-10-30T12:41:25.235 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-30T12:41:25.285 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-30T12:41:25.335 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-30T12:41:25.341 INFO:teuthology.orchestra.run.smithi008.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy InRelease 2023-10-30T12:41:25.393 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-30T12:41:25.402 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-30T12:41:25.460 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-30T12:41:25.465 INFO:teuthology.orchestra.run.smithi008.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-30T12:41:25.527 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-30T12:41:25.608 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.5 kB in 1s (44.6 kB/s) 2023-10-30T12:41:25.664 INFO:teuthology.orchestra.run.smithi008.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-30T12:41:25.818 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 28.5 kB in 1s (48.0 kB/s) 2023-10-30T12:41:26.497 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-30T12:41:26.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6835-g00c1c13c-1jammy cephadm=18.0.0-6835-g00c1c13c-1jammy ceph-mds=18.0.0-6835-g00c1c13c-1jammy ceph-mgr=18.0.0-6835-g00c1c13c-1jammy ceph-common=18.0.0-6835-g00c1c13c-1jammy ceph-fuse=18.0.0-6835-g00c1c13c-1jammy ceph-test=18.0.0-6835-g00c1c13c-1jammy ceph-volume=18.0.0-6835-g00c1c13c-1jammy radosgw=18.0.0-6835-g00c1c13c-1jammy python3-rados=18.0.0-6835-g00c1c13c-1jammy python3-rgw=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy python3-rbd=18.0.0-6835-g00c1c13c-1jammy libcephfs2=18.0.0-6835-g00c1c13c-1jammy libcephfs-dev=18.0.0-6835-g00c1c13c-1jammy librados2=18.0.0-6835-g00c1c13c-1jammy librbd1=18.0.0-6835-g00c1c13c-1jammy rbd-fuse=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy cephfs-shell=18.0.0-6835-g00c1c13c-1jammy cephfs-top=18.0.0-6835-g00c1c13c-1jammy cephfs-mirror=18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:41:26.567 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-30T12:41:26.735 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-10-30T12:41:26.736 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-10-30T12:41:26.760 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-30T12:41:26.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6835-g00c1c13c-1jammy cephadm=18.0.0-6835-g00c1c13c-1jammy ceph-mds=18.0.0-6835-g00c1c13c-1jammy ceph-mgr=18.0.0-6835-g00c1c13c-1jammy ceph-common=18.0.0-6835-g00c1c13c-1jammy ceph-fuse=18.0.0-6835-g00c1c13c-1jammy ceph-test=18.0.0-6835-g00c1c13c-1jammy ceph-volume=18.0.0-6835-g00c1c13c-1jammy radosgw=18.0.0-6835-g00c1c13c-1jammy python3-rados=18.0.0-6835-g00c1c13c-1jammy python3-rgw=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy python3-rbd=18.0.0-6835-g00c1c13c-1jammy libcephfs2=18.0.0-6835-g00c1c13c-1jammy libcephfs-dev=18.0.0-6835-g00c1c13c-1jammy librados2=18.0.0-6835-g00c1c13c-1jammy librbd1=18.0.0-6835-g00c1c13c-1jammy rbd-fuse=18.0.0-6835-g00c1c13c-1jammy python3-cephfs=18.0.0-6835-g00c1c13c-1jammy cephfs-shell=18.0.0-6835-g00c1c13c-1jammy cephfs-top=18.0.0-6835-g00c1c13c-1jammy cephfs-mirror=18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:41:26.852 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-30T12:41:26.889 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:41:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:41:26.889 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:41:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:41:26.890 INFO:teuthology.orchestra.run.smithi046.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-30T12:41:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: python2.7-minimal 2023-10-30T12:41:26.891 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:41:26.891 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-10-30T12:41:26.891 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-30T12:41:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-30T12:41:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:41:26.892 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-30T12:41:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-30T12:41:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-30T12:41:26.893 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-30T12:41:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-30T12:41:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-30T12:41:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-30T12:41:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-30T12:41:26.894 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-30T12:41:26.895 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-30T12:41:26.896 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-10-30T12:41:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-30T12:41:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-30T12:41:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-30T12:41:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-30T12:41:26.897 INFO:teuthology.orchestra.run.smithi046.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-30T12:41:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-30T12:41:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: python3-watchdog 2023-10-30T12:41:26.898 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2023-10-30T12:41:26.898 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2023-10-30T12:41:26.995 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-10-30T12:41:26.995 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-30T12:41:26.996 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-30T12:41:26.996 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-30T12:41:26.996 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-30T12:41:26.997 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:41:26.997 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-30T12:41:26.998 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-30T12:41:26.998 INFO:teuthology.orchestra.run.smithi046.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-30T12:41:26.998 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-30T12:41:26.998 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-30T12:41:26.999 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-30T12:41:26.999 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-30T12:41:26.999 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-30T12:41:27.000 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-30T12:41:27.001 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-30T12:41:27.001 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-30T12:41:27.001 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-30T12:41:27.001 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-30T12:41:27.001 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-30T12:41:27.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-30T12:41:27.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-30T12:41:27.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-30T12:41:27.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-30T12:41:27.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-30T12:41:27.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-30T12:41:27.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-30T12:41:27.003 INFO:teuthology.orchestra.run.smithi046.stdout: socat xmlstarlet 2023-10-30T12:41:27.021 INFO:teuthology.orchestra.run.smithi008.stdout:Building dependency tree... 2023-10-30T12:41:27.021 INFO:teuthology.orchestra.run.smithi008.stdout:Reading state information... 2023-10-30T12:41:27.177 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:41:27.178 INFO:teuthology.orchestra.run.smithi008.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:41:27.178 INFO:teuthology.orchestra.run.smithi008.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:41:27.178 INFO:teuthology.orchestra.run.smithi008.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:41:27.178 INFO:teuthology.orchestra.run.smithi008.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-30T12:41:27.180 INFO:teuthology.orchestra.run.smithi008.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-30T12:41:27.180 INFO:teuthology.orchestra.run.smithi008.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:41:27.180 INFO:teuthology.orchestra.run.smithi008.stdout:The following additional packages will be installed: 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-30T12:41:27.181 INFO:teuthology.orchestra.run.smithi008.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-30T12:41:27.182 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-30T12:41:27.182 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-30T12:41:27.182 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-30T12:41:27.182 INFO:teuthology.orchestra.run.smithi008.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-30T12:41:27.183 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-30T12:41:27.183 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-30T12:41:27.183 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-30T12:41:27.183 INFO:teuthology.orchestra.run.smithi008.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-30T12:41:27.183 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-30T12:41:27.184 INFO:teuthology.orchestra.run.smithi008.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-30T12:41:27.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-30T12:41:27.185 INFO:teuthology.orchestra.run.smithi008.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-30T12:41:27.186 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-30T12:41:27.186 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 184 MB of archives. 2023-10-30T12:41:27.187 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-30T12:41:27.187 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-30T12:41:27.187 INFO:teuthology.orchestra.run.smithi008.stdout:Suggested packages: 2023-10-30T12:41:27.187 INFO:teuthology.orchestra.run.smithi008.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-30T12:41:27.187 INFO:teuthology.orchestra.run.smithi008.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout: python3-watchdog 2023-10-30T12:41:27.188 INFO:teuthology.orchestra.run.smithi008.stdout:Recommended packages: 2023-10-30T12:41:27.189 INFO:teuthology.orchestra.run.smithi008.stdout: btrfs-tools 2023-10-30T12:41:27.194 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6835-g00c1c13c-1jammy [3,450 kB] 2023-10-30T12:41:27.265 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-30T12:41:27.329 INFO:teuthology.orchestra.run.smithi008.stdout:The following NEW packages will be installed: 2023-10-30T12:41:27.329 INFO:teuthology.orchestra.run.smithi008.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-30T12:41:27.329 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-30T12:41:27.330 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-30T12:41:27.331 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-30T12:41:27.331 INFO:teuthology.orchestra.run.smithi008.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-30T12:41:27.331 INFO:teuthology.orchestra.run.smithi008.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-30T12:41:27.332 INFO:teuthology.orchestra.run.smithi008.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-30T12:41:27.333 INFO:teuthology.orchestra.run.smithi008.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-30T12:41:27.334 INFO:teuthology.orchestra.run.smithi008.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-30T12:41:27.335 INFO:teuthology.orchestra.run.smithi008.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-30T12:41:27.335 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-30T12:41:27.335 INFO:teuthology.orchestra.run.smithi008.stdout: socat xmlstarlet 2023-10-30T12:41:27.356 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-30T12:41:27.513 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-30T12:41:27.527 INFO:teuthology.orchestra.run.smithi008.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-30T12:41:27.527 INFO:teuthology.orchestra.run.smithi008.stdout:Need to get 184 MB of archives. 2023-10-30T12:41:27.528 INFO:teuthology.orchestra.run.smithi008.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-30T12:41:27.528 INFO:teuthology.orchestra.run.smithi008.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-30T12:41:27.546 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-30T12:41:27.609 INFO:teuthology.orchestra.run.smithi008.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-30T12:41:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-30T12:41:27.632 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-30T12:41:27.655 INFO:teuthology.orchestra.run.smithi008.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-30T12:41:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-30T12:41:27.754 INFO:teuthology.orchestra.run.smithi008.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6835-g00c1c13c-1jammy [3,450 kB] 2023-10-30T12:41:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-30T12:41:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-30T12:41:27.861 INFO:teuthology.orchestra.run.smithi008.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-30T12:41:27.878 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-30T12:41:27.905 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-30T12:41:27.981 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-30T12:41:27.992 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-30T12:41:28.038 INFO:teuthology.orchestra.run.smithi008.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-30T12:41:28.050 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-30T12:41:28.203 INFO:teuthology.orchestra.run.smithi008.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-30T12:41:28.212 INFO:teuthology.orchestra.run.smithi008.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-30T12:41:28.239 INFO:teuthology.orchestra.run.smithi008.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-30T12:41:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-30T12:41:28.413 INFO:teuthology.orchestra.run.smithi008.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-30T12:41:28.603 INFO:teuthology.orchestra.run.smithi008.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-30T12:41:28.611 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-30T12:41:28.619 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-30T12:41:28.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-30T12:41:28.625 INFO:teuthology.orchestra.run.smithi008.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6835-g00c1c13c-1jammy [3,131 kB] 2023-10-30T12:41:28.632 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-30T12:41:28.633 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-30T12:41:28.635 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-30T12:41:28.637 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-30T12:41:28.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-30T12:41:28.640 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-30T12:41:28.805 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-30T12:41:28.934 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6835-g00c1c13c-1jammy [3,131 kB] 2023-10-30T12:41:29.027 INFO:teuthology.orchestra.run.smithi008.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-30T12:41:29.034 INFO:teuthology.orchestra.run.smithi008.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-30T12:41:29.036 INFO:teuthology.orchestra.run.smithi008.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-30T12:41:29.047 INFO:teuthology.orchestra.run.smithi008.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-30T12:41:29.048 INFO:teuthology.orchestra.run.smithi008.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-30T12:41:29.049 INFO:teuthology.orchestra.run.smithi008.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-30T12:41:29.051 INFO:teuthology.orchestra.run.smithi008.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-30T12:41:29.053 INFO:teuthology.orchestra.run.smithi008.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-30T12:41:29.055 INFO:teuthology.orchestra.run.smithi008.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-30T12:41:29.097 INFO:teuthology.orchestra.run.smithi008.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6835-g00c1c13c-1jammy [689 kB] 2023-10-30T12:41:29.219 INFO:teuthology.orchestra.run.smithi008.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-30T12:41:29.227 INFO:teuthology.orchestra.run.smithi008.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6835-g00c1c13c-1jammy [356 kB] 2023-10-30T12:41:29.236 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-30T12:41:29.246 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-30T12:41:29.252 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-30T12:41:29.255 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-30T12:41:29.273 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-30T12:41:29.277 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-30T12:41:29.279 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-30T12:41:29.296 INFO:teuthology.orchestra.run.smithi008.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6835-g00c1c13c-1jammy [32.8 kB] 2023-10-30T12:41:29.297 INFO:teuthology.orchestra.run.smithi008.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6835-g00c1c13c-1jammy [180 kB] 2023-10-30T12:41:29.304 INFO:teuthology.orchestra.run.smithi008.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6835-g00c1c13c-1jammy [62.4 kB] 2023-10-30T12:41:29.321 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-30T12:41:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-30T12:41:29.339 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-30T12:41:29.363 INFO:teuthology.orchestra.run.smithi008.stdout:Get:31 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6835-g00c1c13c-1jammy [330 kB] 2023-10-30T12:41:29.377 INFO:teuthology.orchestra.run.smithi008.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6835-g00c1c13c-1jammy [6,226 kB] 2023-10-30T12:41:29.592 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-30T12:41:29.646 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6835-g00c1c13c-1jammy [689 kB] 2023-10-30T12:41:29.666 INFO:teuthology.orchestra.run.smithi008.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-30T12:41:29.675 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-30T12:41:29.676 INFO:teuthology.orchestra.run.smithi008.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-30T12:41:29.682 INFO:teuthology.orchestra.run.smithi008.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-30T12:41:29.686 INFO:teuthology.orchestra.run.smithi008.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-30T12:41:29.705 INFO:teuthology.orchestra.run.smithi008.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-30T12:41:29.708 INFO:teuthology.orchestra.run.smithi008.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-30T12:41:29.711 INFO:teuthology.orchestra.run.smithi008.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-30T12:41:29.751 INFO:teuthology.orchestra.run.smithi008.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-30T12:41:29.765 INFO:teuthology.orchestra.run.smithi008.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-30T12:41:29.766 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-30T12:41:29.770 INFO:teuthology.orchestra.run.smithi008.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-30T12:41:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6835-g00c1c13c-1jammy [356 kB] 2023-10-30T12:41:29.822 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-30T12:41:29.865 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6835-g00c1c13c-1jammy [32.8 kB] 2023-10-30T12:41:29.866 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6835-g00c1c13c-1jammy [180 kB] 2023-10-30T12:41:29.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6835-g00c1c13c-1jammy [62.4 kB] 2023-10-30T12:41:29.875 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6835-g00c1c13c-1jammy [330 kB] 2023-10-30T12:41:29.944 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-30T12:41:29.948 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6835-g00c1c13c-1jammy [6,226 kB] 2023-10-30T12:41:30.024 INFO:teuthology.orchestra.run.smithi008.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-30T12:41:30.031 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-30T12:41:30.049 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-30T12:41:30.078 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-30T12:41:30.091 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-30T12:41:30.106 INFO:teuthology.orchestra.run.smithi008.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-30T12:41:30.129 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-30T12:41:30.136 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-30T12:41:30.199 INFO:teuthology.orchestra.run.smithi008.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-30T12:41:30.275 INFO:teuthology.orchestra.run.smithi008.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-30T12:41:30.328 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-30T12:41:30.377 INFO:teuthology.orchestra.run.smithi008.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-30T12:41:30.464 INFO:teuthology.orchestra.run.smithi008.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-30T12:41:30.483 INFO:teuthology.orchestra.run.smithi008.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-30T12:41:30.513 INFO:teuthology.orchestra.run.smithi008.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-30T12:41:30.525 INFO:teuthology.orchestra.run.smithi008.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-30T12:41:30.562 INFO:teuthology.orchestra.run.smithi008.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-30T12:41:30.566 INFO:teuthology.orchestra.run.smithi008.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6835-g00c1c13c-1jammy [112 kB] 2023-10-30T12:41:30.568 INFO:teuthology.orchestra.run.smithi008.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6835-g00c1c13c-1jammy [439 kB] 2023-10-30T12:41:30.570 INFO:teuthology.orchestra.run.smithi008.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-30T12:41:30.634 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6835-g00c1c13c-1jammy [112 kB] 2023-10-30T12:41:30.637 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6835-g00c1c13c-1jammy [439 kB] 2023-10-30T12:41:30.687 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6835-g00c1c13c-1jammy [22.6 MB] 2023-10-30T12:41:30.699 INFO:teuthology.orchestra.run.smithi008.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6835-g00c1c13c-1jammy [22.6 MB] 2023-10-30T12:41:30.761 INFO:teuthology.orchestra.run.smithi008.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-30T12:41:31.141 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-30T12:41:31.143 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-30T12:41:31.146 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-30T12:41:31.147 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-30T12:41:31.148 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-30T12:41:31.151 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-30T12:41:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-30T12:41:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-30T12:41:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-30T12:41:31.253 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-30T12:41:31.418 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-30T12:41:31.593 INFO:teuthology.orchestra.run.smithi008.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-30T12:41:31.594 INFO:teuthology.orchestra.run.smithi008.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-30T12:41:31.596 INFO:teuthology.orchestra.run.smithi008.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-30T12:41:31.596 INFO:teuthology.orchestra.run.smithi008.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-30T12:41:31.597 INFO:teuthology.orchestra.run.smithi008.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-30T12:41:31.598 INFO:teuthology.orchestra.run.smithi008.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-30T12:41:31.621 INFO:teuthology.orchestra.run.smithi008.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-30T12:41:31.624 INFO:teuthology.orchestra.run.smithi008.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-30T12:41:31.624 INFO:teuthology.orchestra.run.smithi008.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-30T12:41:31.676 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-30T12:41:31.695 INFO:teuthology.orchestra.run.smithi008.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-30T12:41:31.708 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-30T12:41:31.734 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-30T12:41:31.786 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-30T12:41:31.806 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-30T12:41:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-30T12:41:31.858 INFO:teuthology.orchestra.run.smithi008.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-30T12:41:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-30T12:41:31.994 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-30T12:41:32.014 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-30T12:41:32.021 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-30T12:41:32.117 INFO:teuthology.orchestra.run.smithi008.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-30T12:41:32.148 INFO:teuthology.orchestra.run.smithi008.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-30T12:41:32.211 INFO:teuthology.orchestra.run.smithi008.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-30T12:41:32.226 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-30T12:41:32.246 INFO:teuthology.orchestra.run.smithi008.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-30T12:41:32.264 INFO:teuthology.orchestra.run.smithi008.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-30T12:41:32.290 INFO:teuthology.orchestra.run.smithi008.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-30T12:41:32.423 INFO:teuthology.orchestra.run.smithi008.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-30T12:41:32.458 INFO:teuthology.orchestra.run.smithi008.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-30T12:41:32.479 INFO:teuthology.orchestra.run.smithi008.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-30T12:41:32.486 INFO:teuthology.orchestra.run.smithi008.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-30T12:41:32.590 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6835-g00c1c13c-1jammy [6,858 kB] 2023-10-30T12:41:32.686 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-30T12:41:32.689 INFO:teuthology.orchestra.run.smithi008.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-30T12:41:32.716 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-30T12:41:32.869 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-30T12:41:32.877 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-30T12:41:32.878 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-30T12:41:32.880 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-30T12:41:32.933 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-30T12:41:32.934 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6835-g00c1c13c-1jammy [238 kB] 2023-10-30T12:41:32.937 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-30T12:41:32.939 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-30T12:41:32.940 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6835-g00c1c13c-1jammy [128 kB] 2023-10-30T12:41:32.942 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6835-g00c1c13c-1jammy [1,496 kB] 2023-10-30T12:41:32.958 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-30T12:41:33.001 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6835-g00c1c13c-1jammy [9,892 kB] 2023-10-30T12:41:33.122 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-30T12:41:33.167 INFO:teuthology.orchestra.run.smithi008.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-30T12:41:33.173 INFO:teuthology.orchestra.run.smithi008.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-30T12:41:33.289 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-30T12:41:33.301 INFO:teuthology.orchestra.run.smithi008.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-30T12:41:33.348 INFO:teuthology.orchestra.run.smithi008.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-30T12:41:33.349 INFO:teuthology.orchestra.run.smithi008.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-30T12:41:33.354 INFO:teuthology.orchestra.run.smithi008.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-30T12:41:33.356 INFO:teuthology.orchestra.run.smithi008.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-30T12:41:33.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-30T12:41:33.380 INFO:teuthology.orchestra.run.smithi008.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-30T12:41:33.440 INFO:teuthology.orchestra.run.smithi008.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-30T12:41:33.571 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-30T12:41:33.571 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6835-g00c1c13c-1jammy [31.7 MB] 2023-10-30T12:41:33.587 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-30T12:41:33.606 INFO:teuthology.orchestra.run.smithi008.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-30T12:41:33.772 INFO:teuthology.orchestra.run.smithi008.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-30T12:41:34.061 INFO:teuthology.orchestra.run.smithi008.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-30T12:41:34.087 INFO:teuthology.orchestra.run.smithi008.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-30T12:41:34.875 INFO:teuthology.orchestra.run.smithi008.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6835-g00c1c13c-1jammy [6,858 kB] 2023-10-30T12:41:35.415 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6835-g00c1c13c-1jammy [14.1 kB] 2023-10-30T12:41:35.416 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [839 kB] 2023-10-30T12:41:35.433 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6835-g00c1c13c-1jammy [2,148 kB] 2023-10-30T12:41:35.478 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6835-g00c1c13c-1jammy [284 kB] 2023-10-30T12:41:35.504 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6835-g00c1c13c-1jammy [138 kB] 2023-10-30T12:41:35.523 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6835-g00c1c13c-1jammy [1,907 kB] 2023-10-30T12:41:35.567 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6835-g00c1c13c-1jammy [8,625 kB] 2023-10-30T12:41:35.983 INFO:teuthology.orchestra.run.smithi008.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6835-g00c1c13c-1jammy [238 kB] 2023-10-30T12:41:36.013 INFO:teuthology.orchestra.run.smithi008.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6835-g00c1c13c-1jammy [128 kB] 2023-10-30T12:41:36.031 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6835-g00c1c13c-1jammy [14.2 kB] 2023-10-30T12:41:36.031 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6835-g00c1c13c-1jammy [47.8 MB] 2023-10-30T12:41:36.036 INFO:teuthology.orchestra.run.smithi008.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6835-g00c1c13c-1jammy [1,496 kB] 2023-10-30T12:41:36.296 INFO:teuthology.orchestra.run.smithi008.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6835-g00c1c13c-1jammy [9,892 kB] 2023-10-30T12:41:37.932 INFO:teuthology.orchestra.run.smithi008.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6835-g00c1c13c-1jammy [31.7 MB] 2023-10-30T12:41:38.808 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6835-g00c1c13c-1jammy [120 kB] 2023-10-30T12:41:38.882 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6835-g00c1c13c-1jammy [227 kB] 2023-10-30T12:41:38.887 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6835-g00c1c13c-1jammy [34.0 kB] 2023-10-30T12:41:38.888 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6835-g00c1c13c-1jammy [23.4 kB] 2023-10-30T12:41:38.889 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6835-g00c1c13c-1jammy [38.3 kB] 2023-10-30T12:41:38.890 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6835-g00c1c13c-1jammy [12.3 MB] 2023-10-30T12:41:39.396 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [91.6 kB] 2023-10-30T12:41:39.972 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 184 MB in 12s (14.9 MB/s) 2023-10-30T12:41:40.119 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-30T12:41:40.205 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 130506 files and directories currently installed.) 2023-10-30T12:41:40.210 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-30T12:41:40.251 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-30T12:41:40.554 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-30T12:41:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-30T12:41:40.627 INFO:teuthology.orchestra.run.smithi008.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6835-g00c1c13c-1jammy [14.1 kB] 2023-10-30T12:41:40.628 INFO:teuthology.orchestra.run.smithi008.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [839 kB] 2023-10-30T12:41:40.629 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:41:40.670 INFO:teuthology.orchestra.run.smithi008.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6835-g00c1c13c-1jammy [2,148 kB] 2023-10-30T12:41:40.782 INFO:teuthology.orchestra.run.smithi008.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6835-g00c1c13c-1jammy [284 kB] 2023-10-30T12:41:40.797 INFO:teuthology.orchestra.run.smithi008.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6835-g00c1c13c-1jammy [138 kB] 2023-10-30T12:41:40.804 INFO:teuthology.orchestra.run.smithi008.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6835-g00c1c13c-1jammy [1,907 kB] 2023-10-30T12:41:40.890 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2023-10-30T12:41:40.908 INFO:teuthology.orchestra.run.smithi008.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6835-g00c1c13c-1jammy [8,625 kB] 2023-10-30T12:41:40.911 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-30T12:41:40.940 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-30T12:41:41.259 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-30T12:41:41.280 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-30T12:41:41.359 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:41:41.389 INFO:teuthology.orchestra.run.smithi008.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6835-g00c1c13c-1jammy [14.2 kB] 2023-10-30T12:41:41.389 INFO:teuthology.orchestra.run.smithi008.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6835-g00c1c13c-1jammy [47.8 MB] 2023-10-30T12:41:41.671 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-30T12:41:41.692 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-30T12:41:41.721 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:41:42.049 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-30T12:41:42.070 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-30T12:41:42.108 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:41:42.461 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-30T12:41:42.481 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-30T12:41:42.519 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:41:42.931 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-30T12:41:42.952 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:42.989 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:43.326 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-30T12:41:43.347 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:43.376 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:43.722 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-30T12:41:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:43.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:44.084 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-30T12:41:44.089 INFO:teuthology.orchestra.run.smithi008.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6835-g00c1c13c-1jammy [120 kB] 2023-10-30T12:41:44.090 INFO:teuthology.orchestra.run.smithi008.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6835-g00c1c13c-1jammy [227 kB] 2023-10-30T12:41:44.092 INFO:teuthology.orchestra.run.smithi008.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6835-g00c1c13c-1jammy [34.0 kB] 2023-10-30T12:41:44.093 INFO:teuthology.orchestra.run.smithi008.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6835-g00c1c13c-1jammy [23.4 kB] 2023-10-30T12:41:44.093 INFO:teuthology.orchestra.run.smithi008.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6835-g00c1c13c-1jammy [38.3 kB] 2023-10-30T12:41:44.096 INFO:teuthology.orchestra.run.smithi008.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6835-g00c1c13c-1jammy [12.3 MB] 2023-10-30T12:41:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-30T12:41:44.133 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:41:44.437 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2023-10-30T12:41:44.458 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-librados2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:44.495 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:44.802 INFO:teuthology.orchestra.run.smithi008.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-lusov-balance-after-stop/00c1c13cea21a686b046584596fbeac5315a7d71/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6835-g00c1c13c-1jammy [91.6 kB] 2023-10-30T12:41:45.181 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2023-10-30T12:41:45.202 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-librbd1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:45.239 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:45.398 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 184 MB in 17s (10.6 MB/s) 2023-10-30T12:41:45.551 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-30T12:41:45.623 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-30T12:41:45.626 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-30T12:41:45.642 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2023-10-30T12:41:45.663 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:45.683 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-30T12:41:45.700 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:45.936 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2023-10-30T12:41:45.949 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:46.011 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:46.094 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-30T12:41:46.108 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-30T12:41:46.152 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:41:46.331 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-30T12:41:46.352 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:41:46.389 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:46.439 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cmd2. 2023-10-30T12:41:46.453 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-30T12:41:46.497 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-30T12:41:46.642 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2023-10-30T12:41:46.662 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:46.699 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:46.858 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-30T12:41:46.874 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-30T12:41:46.958 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:41:46.961 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-30T12:41:46.981 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:41:47.010 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:47.228 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-30T12:41:47.243 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-30T12:41:47.286 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:41:47.289 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2023-10-30T12:41:47.310 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-30T12:41:47.347 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-30T12:41:47.600 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2023-10-30T12:41:47.620 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:47.632 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-30T12:41:47.647 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-30T12:41:47.658 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:47.690 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:41:48.019 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-30T12:41:48.040 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-30T12:41:48.052 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-30T12:41:48.066 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-30T12:41:48.077 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:41:48.109 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:41:48.406 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2023-10-30T12:41:48.426 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-librgw2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:48.464 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:48.480 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-30T12:41:48.495 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:48.538 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:48.950 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-30T12:41:48.958 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2023-10-30T12:41:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:48.979 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:49.008 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:49.016 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:49.361 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-30T12:41:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-30T12:41:49.382 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-30T12:41:49.393 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-30T12:41:49.419 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:41:49.436 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:41:49.722 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2023-10-30T12:41:49.742 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:49.748 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-30T12:41:49.763 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-30T12:41:49.788 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:49.806 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:41:50.126 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librados2. 2023-10-30T12:41:50.140 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../011-librados2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:50.258 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2023-10-30T12:41:50.280 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:50.325 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:50.359 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:50.753 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librbd1. 2023-10-30T12:41:50.768 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../012-librbd1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:50.811 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:51.215 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libcephfs2. 2023-10-30T12:41:51.230 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:51.264 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:51.542 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rados. 2023-10-30T12:41:51.557 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:51.600 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:51.647 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2023-10-30T12:41:51.668 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:51.716 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:51.920 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-30T12:41:51.935 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:41:51.978 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:52.239 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cephfs. 2023-10-30T12:41:52.254 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:52.297 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:52.435 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-30T12:41:52.455 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-30T12:41:52.493 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:41:52.600 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-30T12:41:52.615 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:41:52.667 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:52.812 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2023-10-30T12:41:52.833 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-30T12:41:52.870 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:41:52.962 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-prettytable. 2023-10-30T12:41:52.977 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-30T12:41:53.019 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-30T12:41:53.182 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-30T12:41:53.203 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-30T12:41:53.240 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:41:53.264 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rbd. 2023-10-30T12:41:53.279 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:53.322 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:53.518 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-30T12:41:53.539 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-30T12:41:53.576 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:41:53.661 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-30T12:41:53.676 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-30T12:41:53.719 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:41:53.879 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-30T12:41:53.900 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-30T12:41:53.937 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:41:54.047 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librgw2. 2023-10-30T12:41:54.062 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../021-librgw2_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:54.105 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:54.240 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2023-10-30T12:41:54.261 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-30T12:41:54.299 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-30T12:41:54.566 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rgw. 2023-10-30T12:41:54.568 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2023-10-30T12:41:54.581 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-30T12:41:54.582 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:54.609 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-30T12:41:54.624 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:54.904 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-30T12:41:54.925 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-30T12:41:54.962 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-30T12:41:54.994 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-30T12:41:55.009 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-30T12:41:55.052 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:41:55.339 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libradosstriper1. 2023-10-30T12:41:55.354 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:55.397 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:55.424 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-30T12:41:55.445 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-30T12:41:55.482 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:41:55.836 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2023-10-30T12:41:55.842 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-common. 2023-10-30T12:41:55.857 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-30T12:41:55.857 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:55.894 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-30T12:41:55.900 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:56.225 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2023-10-30T12:41:56.246 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-30T12:41:56.288 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-30T12:41:56.550 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-30T12:41:56.570 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-30T12:41:56.600 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:41:56.877 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2023-10-30T12:41:56.898 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-30T12:41:56.936 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:41:56.981 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-base. 2023-10-30T12:41:56.997 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:41:57.041 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:41:57.314 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-30T12:41:57.335 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-30T12:41:57.372 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:41:57.667 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-30T12:41:57.688 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-30T12:41:57.694 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-30T12:41:57.708 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-30T12:41:57.725 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:41:57.752 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:41:57.986 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2023-10-30T12:41:58.007 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-30T12:41:58.036 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:41:58.055 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cheroot. 2023-10-30T12:41:58.070 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-30T12:41:58.114 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:41:58.389 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2023-10-30T12:41:58.410 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-30T12:41:58.425 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-30T12:41:58.441 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-30T12:41:58.458 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:41:58.509 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:41:58.717 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2023-10-30T12:41:58.738 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-30T12:41:58.770 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-30T12:41:58.775 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:41:58.785 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-30T12:41:58.820 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:41:59.078 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2023-10-30T12:41:59.099 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-30T12:41:59.123 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-30T12:41:59.136 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:41:59.138 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-30T12:41:59.181 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:41:59.423 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-30T12:41:59.435 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-30T12:41:59.473 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:41:59.484 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-tempora. 2023-10-30T12:41:59.499 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-30T12:41:59.542 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-30T12:41:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-30T12:41:59.789 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-30T12:41:59.825 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:41:59.845 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-portend. 2023-10-30T12:41:59.860 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-30T12:41:59.895 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-30T12:42:00.129 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2023-10-30T12:42:00.150 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-30T12:42:00.195 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-30T12:42:00.498 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2023-10-30T12:42:00.519 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-30T12:42:00.556 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:42:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-30T12:42:00.839 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-30T12:42:00.882 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-30T12:42:00.886 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2023-10-30T12:42:00.907 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-30T12:42:00.936 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-30T12:42:01.193 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-30T12:42:01.208 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-30T12:42:01.247 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2023-10-30T12:42:01.251 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:42:01.268 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-30T12:42:01.305 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-30T12:42:01.614 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-natsort. 2023-10-30T12:42:01.625 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2023-10-30T12:42:01.629 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-30T12:42:01.646 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-30T12:42:01.672 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-30T12:42:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:42:02.008 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-logutils. 2023-10-30T12:42:02.020 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2023-10-30T12:42:02.023 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-30T12:42:02.041 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-30T12:42:02.075 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-30T12:42:02.086 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:42:02.370 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-30T12:42:02.385 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-30T12:42:02.406 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-30T12:42:02.427 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-30T12:42:02.428 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:42:02.464 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:42:02.672 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-mako. 2023-10-30T12:42:02.688 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-30T12:42:02.730 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:42:02.759 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-30T12:42:02.771 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:02.809 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:03.051 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-30T12:42:03.067 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-30T12:42:03.126 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:42:03.213 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-30T12:42:03.235 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:03.288 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:03.429 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-30T12:42:03.445 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-30T12:42:03.487 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:42:03.608 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2023-10-30T12:42:03.629 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-30T12:42:03.666 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-30T12:42:03.773 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-webob. 2023-10-30T12:42:03.789 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-30T12:42:03.831 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:42:04.011 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2023-10-30T12:42:04.032 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:04.069 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:04.185 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-waitress. 2023-10-30T12:42:04.200 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-30T12:42:04.271 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:42:04.414 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2023-10-30T12:42:04.435 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:04.472 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:04.546 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-tempita. 2023-10-30T12:42:04.562 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-30T12:42:04.604 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:42:04.907 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-paste. 2023-10-30T12:42:04.923 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-30T12:42:04.965 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:42:05.311 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-30T12:42:05.326 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-30T12:42:05.369 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:42:05.376 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2023-10-30T12:42:05.397 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:05.590 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:05.688 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-30T12:42:05.704 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-30T12:42:05.746 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:42:06.033 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-webtest. 2023-10-30T12:42:06.048 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-30T12:42:06.091 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-30T12:42:06.378 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pecan. 2023-10-30T12:42:06.393 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-30T12:42:06.427 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:42:06.766 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-certifi. 2023-10-30T12:42:06.782 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-30T12:42:06.816 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-30T12:42:07.144 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-idna. 2023-10-30T12:42:07.159 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-30T12:42:07.202 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-30T12:42:07.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2023-10-30T12:42:07.483 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:07.513 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-urllib3. 2023-10-30T12:42:07.520 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:07.529 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-30T12:42:07.572 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:42:07.831 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2023-10-30T12:42:07.853 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:07.889 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:07.934 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-requests. 2023-10-30T12:42:07.949 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-30T12:42:07.992 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:42:08.251 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2023-10-30T12:42:08.272 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:08.303 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-30T12:42:08.309 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:08.319 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-30T12:42:08.362 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:42:08.657 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-30T12:42:08.672 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:08.687 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2023-10-30T12:42:08.708 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-cephadm_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:08.714 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:08.745 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:09.057 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-30T12:42:09.078 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-30T12:42:09.115 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-30T12:42:09.145 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-30T12:42:09.160 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:09.203 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:09.401 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-30T12:42:09.413 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:42:09.451 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:09.498 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-openssl. 2023-10-30T12:42:09.513 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-30T12:42:09.555 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-30T12:42:09.892 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr. 2023-10-30T12:42:09.907 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:09.942 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:10.345 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mon. 2023-10-30T12:42:10.361 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:10.404 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:10.406 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2023-10-30T12:42:10.427 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:42:10.464 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:10.943 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2023-10-30T12:42:10.964 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-30T12:42:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-30T12:42:11.354 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-30T12:42:11.374 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-osd. 2023-10-30T12:42:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:11.389 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:11.412 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:11.432 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:11.791 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-30T12:42:11.812 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-30T12:42:11.849 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:42:12.135 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-30T12:42:12.157 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-30T12:42:12.193 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-30T12:42:12.496 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2023-10-30T12:42:12.518 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-30T12:42:12.554 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:42:12.703 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph. 2023-10-30T12:42:12.718 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../061-ceph_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:12.752 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:12.908 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-30T12:42:12.929 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:12.966 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:13.072 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-fuse. 2023-10-30T12:42:13.087 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:13.130 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:13.345 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-30T12:42:13.357 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-30T12:42:13.386 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:42:13.492 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mds. 2023-10-30T12:42:13.507 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:13.567 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:13.902 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2023-10-30T12:42:13.912 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package cephadm. 2023-10-30T12:42:13.923 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-30T12:42:13.927 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../064-cephadm_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:13.962 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:13.973 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:42:14.273 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-30T12:42:14.289 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-30T12:42:14.352 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-30T12:42:14.356 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-30T12:42:14.365 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-30T12:42:14.402 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:42:14.643 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-30T12:42:14.658 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:42:14.704 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2023-10-30T12:42:14.718 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:14.726 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-30T12:42:14.771 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:42:15.480 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-30T12:42:15.502 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:15.546 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:15.782 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-babel. 2023-10-30T12:42:15.796 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-30T12:42:15.865 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:16.202 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jinja2. 2023-10-30T12:42:16.217 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-30T12:42:16.251 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-30T12:42:16.500 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2023-10-30T12:42:16.521 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-30T12:42:16.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-30T12:42:16.588 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-30T12:42:16.603 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:16.646 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:16.861 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-30T12:42:16.882 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-30T12:42:16.927 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:42:17.050 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-30T12:42:17.066 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-30T12:42:17.108 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:42:17.264 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-30T12:42:17.285 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-30T12:42:17.330 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:42:17.403 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-30T12:42:17.419 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-30T12:42:17.461 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-30T12:42:17.642 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2023-10-30T12:42:17.663 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-30T12:42:17.708 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-30T12:42:17.731 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-routes. 2023-10-30T12:42:17.747 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-30T12:42:17.789 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:42:18.147 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2023-10-30T12:42:18.168 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-30T12:42:18.185 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-30T12:42:18.200 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:18.204 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-30T12:42:18.243 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:18.500 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-30T12:42:18.522 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-30T12:42:18.566 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:42:18.663 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-30T12:42:18.678 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-30T12:42:18.721 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:42:18.878 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2023-10-30T12:42:18.900 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-30T12:42:18.944 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-30T12:42:19.192 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-joblib. 2023-10-30T12:42:19.207 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-30T12:42:19.242 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:42:19.264 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-30T12:42:19.286 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-30T12:42:19.330 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:42:19.621 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-30T12:42:19.636 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-30T12:42:19.679 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:42:19.965 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-sklearn. 2023-10-30T12:42:19.981 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-30T12:42:20.023 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:42:20.095 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-30T12:42:20.108 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:20.136 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:20.456 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-30T12:42:20.477 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-30T12:42:20.522 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:42:20.699 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-30T12:42:20.713 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:20.757 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:20.825 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-30T12:42:20.847 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:42:20.883 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:42:21.311 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2023-10-30T12:42:21.333 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:42:21.369 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-30T12:42:21.614 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2023-10-30T12:42:21.627 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-30T12:42:21.672 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:42:21.769 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cachetools. 2023-10-30T12:42:21.785 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-30T12:42:21.836 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-30T12:42:21.986 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2023-10-30T12:42:22.007 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-30T12:42:22.043 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:42:22.114 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-30T12:42:22.130 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-30T12:42:22.172 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:42:22.406 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2023-10-30T12:42:22.428 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:22.472 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:22.491 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-30T12:42:22.507 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-30T12:42:22.550 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:42:22.895 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rsa. 2023-10-30T12:42:22.910 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-30T12:42:22.953 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-30T12:42:23.333 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-google-auth. 2023-10-30T12:42:23.349 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-30T12:42:23.391 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-30T12:42:23.695 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-30T12:42:23.711 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-30T12:42:23.753 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:42:24.090 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-websocket. 2023-10-30T12:42:24.106 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-30T12:42:24.148 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-30T12:42:24.468 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-30T12:42:24.484 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-30T12:42:24.526 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:42:24.726 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2023-10-30T12:42:24.748 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:24.784 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:25.147 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-30T12:42:25.169 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:25.248 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-30T12:42:25.263 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:25.307 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:25.508 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2023-10-30T12:42:25.530 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-30T12:42:25.575 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-30T12:42:25.634 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-30T12:42:25.649 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-30T12:42:25.693 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:42:25.828 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2023-10-30T12:42:25.850 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:25.894 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:26.013 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-30T12:42:26.029 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:42:26.071 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:42:26.163 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2023-10-30T12:42:26.176 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:26.213 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:26.466 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-30T12:42:26.488 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:26.532 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:26.582 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package jq. 2023-10-30T12:42:26.598 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-30T12:42:26.640 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-30T12:42:26.894 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2023-10-30T12:42:26.902 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package socat. 2023-10-30T12:42:26.916 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-30T12:42:26.918 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-30T12:42:26.960 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:42:26.960 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:42:27.238 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package xmlstarlet. 2023-10-30T12:42:27.254 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-30T12:42:27.282 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-30T12:42:27.288 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:42:27.295 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-30T12:42:27.331 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:42:27.676 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-test. 2023-10-30T12:42:27.691 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:27.734 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:27.743 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-30T12:42:27.765 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-30T12:42:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-30T12:42:28.113 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2023-10-30T12:42:28.134 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-30T12:42:28.179 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-30T12:42:28.499 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2023-10-30T12:42:28.521 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-30T12:42:28.565 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:42:28.844 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2023-10-30T12:42:28.866 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-30T12:42:28.910 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-30T12:42:29.238 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2023-10-30T12:42:29.257 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-30T12:42:29.296 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-30T12:42:29.575 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2023-10-30T12:42:29.588 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-30T12:42:29.605 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-volume. 2023-10-30T12:42:29.616 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:42:29.621 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:29.655 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:30.001 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-30T12:42:30.017 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:30.051 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:30.054 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-30T12:42:30.067 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-30T12:42:30.112 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:42:30.338 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-colorama. 2023-10-30T12:42:30.354 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-30T12:42:30.396 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-30T12:42:30.423 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2023-10-30T12:42:30.445 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-30T12:42:30.489 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-30T12:42:30.657 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package cephfs-shell. 2023-10-30T12:42:30.673 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:30.715 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:30.792 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2023-10-30T12:42:30.815 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-30T12:42:30.850 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:42:30.985 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package cephfs-top. 2023-10-30T12:42:31.001 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6835-g00c1c13c-1jammy_all.deb ... 2023-10-30T12:42:31.043 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:31.221 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2023-10-30T12:42:31.243 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-30T12:42:31.298 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-30T12:42:31.314 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:31.356 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:31.375 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:42:31.734 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package nvme-cli. 2023-10-30T12:42:31.750 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-30T12:42:31.773 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-30T12:42:31.786 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-30T12:42:31.793 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:42:31.823 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:42:32.148 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-30T12:42:32.163 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-30T12:42:32.206 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:42:32.502 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2023-10-30T12:42:32.524 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-radosgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:32.568 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:32.617 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-30T12:42:32.632 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-30T12:42:32.675 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-30T12:42:32.962 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pastescript. 2023-10-30T12:42:32.978 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-30T12:42:33.020 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-30T12:42:33.355 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2023-10-30T12:42:33.357 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pluggy. 2023-10-30T12:42:33.368 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:33.373 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-30T12:42:33.405 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:33.415 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:42:33.701 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-psutil. 2023-10-30T12:42:33.707 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-30T12:42:33.717 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-30T12:42:33.759 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-30T12:42:33.976 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:42:34.071 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-py. 2023-10-30T12:42:34.087 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-30T12:42:34.111 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:42:34.129 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-30T12:42:34.376 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-30T12:42:34.474 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pygments. 2023-10-30T12:42:34.490 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-30T12:42:34.532 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:42:34.727 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-30T12:42:34.953 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-30T12:42:34.968 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-30T12:42:35.003 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:42:35.184 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-30T12:42:35.287 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:35.306 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-toml. 2023-10-30T12:42:35.322 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-30T12:42:35.364 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-30T12:42:35.692 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pytest. 2023-10-30T12:42:35.708 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-30T12:42:35.750 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:42:35.821 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2023-10-30T12:42:35.976 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:42:36.104 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-simplejson. 2023-10-30T12:42:36.120 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-30T12:42:36.162 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:42:36.261 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:42:36.432 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-30T12:42:36.448 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-30T12:42:36.482 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:42:36.529 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:42:36.677 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:42:36.939 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-30T12:42:37.199 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:42:37.202 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package radosgw. 2023-10-30T12:42:37.217 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../113-radosgw_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:37.261 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:37.316 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-30T12:42:37.601 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:42:37.855 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package rbd-fuse. 2023-10-30T12:42:37.871 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6835-g00c1c13c-1jammy_amd64.deb ... 2023-10-30T12:42:37.914 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:37.961 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-30T12:42:38.221 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-30T12:42:38.234 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-30T12:42:38.480 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-30T12:42:38.636 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-30T12:42:38.757 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:42:38.761 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-30T12:42:39.024 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:42:39.040 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-30T12:42:39.275 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:42:39.430 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-30T12:42:39.560 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:39.836 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:42:39.962 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-30T12:42:40.022 INFO:teuthology.orchestra.run.smithi008.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-30T12:42:40.078 INFO:teuthology.orchestra.run.smithi008.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-30T12:42:40.130 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:40.229 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:42:40.489 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:42:40.615 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-30T12:42:40.676 INFO:teuthology.orchestra.run.smithi008.stdout:Adding system user cephadm....done 2023-10-30T12:42:40.836 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-30T12:42:40.983 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-30T12:42:41.196 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-30T12:42:41.352 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:42:41.473 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-30T12:42:41.770 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:42:41.842 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-30T12:42:42.087 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:42:42.125 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-30T12:42:42.401 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-30T12:42:42.527 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-30T12:42:42.781 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:42:42.879 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-30T12:42:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:42:43.150 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:42:43.256 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-30T12:42:43.259 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:42:43.533 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-30T12:42:43.560 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-30T12:42:43.809 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-30T12:42:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-30T12:42:44.110 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-30T12:42:44.204 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:42:44.436 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-30T12:42:44.581 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:42:44.712 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-30T12:42:44.882 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-30T12:42:44.988 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-ceph-argparse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:45.175 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:42:45.281 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-30T12:42:45.407 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-30T12:42:45.518 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:42:45.675 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-30T12:42:45.778 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:42:45.904 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:45.943 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-30T12:42:46.102 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-30T12:42:46.222 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1) ... 2023-10-30T12:42:46.487 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:46.548 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:42:46.613 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-30T12:42:46.889 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-30T12:42:47.201 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-30T12:42:47.525 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-30T12:42:47.762 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:42:47.896 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-30T12:42:48.180 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:42:48.244 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-30T12:42:48.306 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:42:48.432 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:42:48.553 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-30T12:42:48.687 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-30T12:42:48.725 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:42:48.813 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-30T12:42:48.985 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-30T12:42:49.131 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-30T12:42:49.245 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:42:49.379 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:42:49.505 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:42:49.541 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-30T12:42:49.808 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-30T12:42:49.814 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:42:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-30T12:42:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-30T12:42:50.333 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-30T12:42:50.520 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-30T12:42:50.576 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:42:50.852 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-30T12:42:50.855 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-30T12:42:51.128 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:42:51.215 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-30T12:42:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-30T12:42:51.482 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-30T12:42:51.547 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:42:51.608 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:51.674 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:42:51.926 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-idna (3.3-1) ... 2023-10-30T12:42:51.975 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:42:52.269 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-30T12:42:52.278 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:42:52.527 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:53.009 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-30T12:42:53.296 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:42:53.422 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:42:53.559 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:42:53.693 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-30T12:42:53.832 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:42:53.969 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-30T12:42:54.083 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-30T12:42:54.120 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-30T12:42:54.221 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-30T12:42:54.317 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:42:54.443 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-30T12:42:54.539 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-30T12:42:54.769 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:42:54.824 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-30T12:42:55.071 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:42:55.091 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:42:55.217 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-30T12:42:55.355 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-30T12:42:55.369 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-30T12:42:55.482 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-30T12:42:55.695 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-30T12:42:55.757 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-30T12:42:55.963 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-30T12:42:56.025 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:42:56.231 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-30T12:42:56.284 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:42:56.507 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-30T12:42:56.669 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-30T12:42:56.775 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-30T12:42:56.979 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:42:57.126 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-30T12:42:57.252 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-30T12:42:57.348 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:42:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:57.566 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:57.570 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-30T12:42:57.696 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-30T12:42:57.786 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:57.997 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-30T12:42:58.038 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:58.164 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-30T12:42:58.315 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-30T12:42:58.448 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:58.558 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:58.583 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-30T12:42:58.667 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:58.894 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:42:58.894 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:42:58.977 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-30T12:42:59.261 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-30T12:42:59.387 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-30T12:42:59.468 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:59.636 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:59.773 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:42:59.814 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-30T12:42:59.905 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:43:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-30T12:43:00.391 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-30T12:43:00.517 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-30T12:43:00.860 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-30T12:43:01.177 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.194 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-30T12:43:01.303 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.429 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.479 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-30T12:43:01.555 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.613 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-30T12:43:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.807 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:01.898 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-30T12:43:02.208 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-30T12:43:02.358 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2023-10-30T12:43:02.483 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-30T12:43:02.735 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2023-10-30T12:43:02.886 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2023-10-30T12:43:02.893 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-30T12:43:02.904 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-30T12:43:03.047 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-30T12:43:03.195 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-30T12:43:03.411 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-30T12:43:03.555 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-30T12:43:03.665 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librados2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:03.791 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librgw2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:03.933 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.002 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.129 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libcephfs2 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.129 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:43:04.129 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:43:04.254 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libradosstriper1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.389 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-30T12:43:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.717 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librbd1 (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.794 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.842 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-modules-core (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:04.968 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:05.190 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:43:05.191 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-30T12:43:05.302 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:43:05.303 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:43:05.786 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libcephfs-dev (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:05.838 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:05.912 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rados (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:06.038 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up cephfs-top (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:06.148 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-30T12:43:06.212 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-30T12:43:06.752 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:07.049 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:43:07.050 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:43:07.486 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rbd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:07.566 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:07.612 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up rbd-fuse (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:07.738 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:07.794 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:43:07.794 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:43:07.865 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cephfs (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:08.016 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up cephfs-shell (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:08.142 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-common (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:08.343 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:08.671 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:43:08.671 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:43:08.696 INFO:teuthology.orchestra.run.smithi008.stdout:Adding group ceph....done 2023-10-30T12:43:09.047 INFO:teuthology.orchestra.run.smithi008.stdout:Adding system user ceph....done 2023-10-30T12:43:09.236 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:09.241 INFO:teuthology.orchestra.run.smithi008.stdout:Setting system user ceph properties....done 2023-10-30T12:43:09.261 INFO:teuthology.orchestra.run.smithi008.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-30T12:43:09.362 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:09.413 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-30T12:43:09.513 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:09.715 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:43:09.715 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:43:09.814 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-30T12:43:10.248 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:10.374 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:10.443 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up cephfs-mirror (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:10.508 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:10.659 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:43:10.659 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-30T12:43:10.659 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:11.013 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:43:11.295 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-test (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:11.454 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up radosgw (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:11.778 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:43:11.995 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:43:11.995 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-30T12:43:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:43:12.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-30T12:43:12.355 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-30T12:43:12.546 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-10-30T12:43:12.547 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-10-30T12:43:12.673 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-base (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:12.718 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-30T12:43:12.719 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-30T12:43:12.719 INFO:teuthology.orchestra.run.smithi046.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:43:12.719 INFO:teuthology.orchestra.run.smithi046.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:43:12.719 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-30T12:43:12.720 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:43:12.720 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:43:12.720 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:43:12.721 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:43:12.721 INFO:teuthology.orchestra.run.smithi046.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-30T12:43:12.722 INFO:teuthology.orchestra.run.smithi046.stdout: python2.7-minimal 2023-10-30T12:43:12.722 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:43:12.722 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-10-30T12:43:12.722 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-30T12:43:12.722 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-30T12:43:12.723 INFO:teuthology.orchestra.run.smithi046.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-30T12:43:12.723 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-30T12:43:12.723 INFO:teuthology.orchestra.run.smithi046.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:43:12.723 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-10-30T12:43:12.724 INFO:teuthology.orchestra.run.smithi046.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-30T12:43:12.724 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-30T12:43:12.724 INFO:teuthology.orchestra.run.smithi046.stdout: libengine-pkcs11-openssl 2023-10-30T12:43:12.760 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-10-30T12:43:12.761 INFO:teuthology.orchestra.run.smithi046.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-30T12:43:12.762 INFO:teuthology.orchestra.run.smithi046.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-30T12:43:12.762 INFO:teuthology.orchestra.run.smithi046.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-30T12:43:12.763 INFO:teuthology.orchestra.run.smithi046.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-30T12:43:12.763 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-30T12:43:12.763 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-30T12:43:12.763 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:43:12.764 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2023-10-30T12:43:12.764 INFO:teuthology.orchestra.run.smithi046.stdout: libssl3 2023-10-30T12:43:12.820 INFO:teuthology.orchestra.run.smithi046.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-30T12:43:12.820 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 54.6 MB of archives. 2023-10-30T12:43:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-30T12:43:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-30T12:43:12.919 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-30T12:43:12.981 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-30T12:43:12.996 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-30T12:43:12.997 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-30T12:43:12.999 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-30T12:43:13.000 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-30T12:43:13.001 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-30T12:43:13.002 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-30T12:43:13.007 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-30T12:43:13.008 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-30T12:43:13.013 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-30T12:43:13.020 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-30T12:43:13.038 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-30T12:43:13.324 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-30T12:43:13.326 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-30T12:43:13.326 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-30T12:43:13.328 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-30T12:43:13.328 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-30T12:43:13.332 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-30T12:43:13.334 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-30T12:43:13.340 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-30T12:43:13.340 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-30T12:43:13.359 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-30T12:43:13.360 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-30T12:43:13.369 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-30T12:43:13.390 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-30T12:43:13.393 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-30T12:43:13.394 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-30T12:43:13.394 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-30T12:43:13.395 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-30T12:43:13.406 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-30T12:43:13.407 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-30T12:43:13.616 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-30T12:43:13.617 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-30T12:43:13.617 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-30T12:43:13.617 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-30T12:43:13.617 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-30T12:43:13.618 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-30T12:43:13.628 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-30T12:43:13.628 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-30T12:43:13.700 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mds (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:13.929 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:43:13.929 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-30T12:43:14.042 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2023-10-30T12:43:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 54.6 MB in 1s (63.7 MB/s) 2023-10-30T12:43:14.411 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package flex. 2023-10-30T12:43:14.493 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137425 files and directories currently installed.) 2023-10-30T12:43:14.507 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:14.551 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-30T12:43:14.704 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:43:14.705 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-30T12:43:15.113 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:43:15.283 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-30T12:43:15.295 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-osd (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:15.627 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:43:15.627 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-30T12:43:15.682 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:16.201 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package bison. 2023-10-30T12:43:16.222 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:16.291 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137498 files and directories currently installed.) 2023-10-30T12:43:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-30T12:43:16.336 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:43:16.356 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:16.507 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mon (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:16.718 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:43:16.718 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-30T12:43:16.754 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-30T12:43:16.776 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-30T12:43:16.820 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:43:17.157 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2023-10-30T12:43:17.179 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-30T12:43:17.215 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b47-3) ... 2023-10-30T12:43:17.309 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-cephadm (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:17.485 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2023-10-30T12:43:17.498 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:17.535 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:43:17.655 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:17.787 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-30T12:43:17.801 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-30T12:43:17.845 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:43:18.015 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-dashboard (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:18.166 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-volume (18.0.0-6835-g00c1c13c-1jammy) ... 2023-10-30T12:43:18.216 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-30T12:43:18.238 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-30T12:43:18.282 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:43:18.519 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:43:18.687 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-30T12:43:18.709 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-30T12:43:18.753 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:43:19.073 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-30T12:43:19.095 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:19.140 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:19.384 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-30T12:43:19.406 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:19.450 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:19.712 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:43:19.720 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjson-perl. 2023-10-30T12:43:19.742 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-30T12:43:19.778 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-30T12:43:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-30T12:43:20.095 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-30T12:43:20.139 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:43:20.193 INFO:teuthology.orchestra.run.smithi008.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:43:20.198 DEBUG:teuthology.orchestra.run.smithi008:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-30T12:43:20.276 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-30T12:43:20.446 INFO:teuthology.orchestra.run.smithi008.stdout:Building dependency tree... 2023-10-30T12:43:20.446 INFO:teuthology.orchestra.run.smithi008.stdout:Reading state information... 2023-10-30T12:43:20.600 INFO:teuthology.orchestra.run.smithi008.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-30T12:43:20.600 INFO:teuthology.orchestra.run.smithi008.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-30T12:43:20.601 INFO:teuthology.orchestra.run.smithi008.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:43:20.601 INFO:teuthology.orchestra.run.smithi008.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-30T12:43:20.601 INFO:teuthology.orchestra.run.smithi008.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-30T12:43:20.601 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages were automatically installed and are no longer required: 2023-10-30T12:43:20.601 INFO:teuthology.orchestra.run.smithi008.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-30T12:43:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-30T12:43:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-30T12:43:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-30T12:43:20.603 INFO:teuthology.orchestra.run.smithi008.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-30T12:43:20.604 INFO:teuthology.orchestra.run.smithi008.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-30T12:43:20.604 INFO:teuthology.orchestra.run.smithi008.stdout:The following additional packages will be installed: 2023-10-30T12:43:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-30T12:43:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-30T12:43:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-30T12:43:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-30T12:43:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:43:20.605 INFO:teuthology.orchestra.run.smithi008.stdout:Suggested packages: 2023-10-30T12:43:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-30T12:43:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-30T12:43:20.606 INFO:teuthology.orchestra.run.smithi008.stdout: libengine-pkcs11-openssl 2023-10-30T12:43:20.654 INFO:teuthology.orchestra.run.smithi008.stdout:The following NEW packages will be installed: 2023-10-30T12:43:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-30T12:43:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-30T12:43:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-30T12:43:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-30T12:43:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-30T12:43:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-30T12:43:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-30T12:43:20.657 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages will be upgraded: 2023-10-30T12:43:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: libssl3 2023-10-30T12:43:21.876 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-30T12:43:21.898 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:43:21.942 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:22.203 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-30T12:43:22.216 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:43:22.252 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:22.555 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-30T12:43:22.577 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:43:22.630 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:22.975 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-30T12:43:22.998 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-30T12:43:23.041 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:43:23.344 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-30T12:43:23.366 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:43:23.410 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:43:23.722 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-30T12:43:23.744 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:23.839 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:24.217 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-30T12:43:24.239 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:24.283 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:24.586 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-30T12:43:24.608 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:24.653 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:24.880 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-30T12:43:24.893 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:43:24.921 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:25.434 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-30T12:43:25.456 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-30T12:43:25.500 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:43:25.845 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2023-10-30T12:43:25.867 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:43:25.911 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2023-10-30T12:43:26.540 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:43:26.584 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:43:27.149 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2023-10-30T12:43:27.172 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-30T12:43:27.466 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:43:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2023-10-30T12:43:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-30T12:43:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-30T12:43:28.098 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-30T12:43:28.113 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-30T12:43:28.139 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-30T12:43:28.418 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-30T12:43:28.440 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-30T12:43:28.484 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-common (238) ... 2023-10-30T12:43:28.789 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-30T12:43:28.810 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:28.816 INFO:teuthology.orchestra.run.smithi008.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-30T12:43:28.816 INFO:teuthology.orchestra.run.smithi008.stdout:Need to get 54.6 MB of archives. 2023-10-30T12:43:28.816 INFO:teuthology.orchestra.run.smithi008.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-30T12:43:28.816 INFO:teuthology.orchestra.run.smithi008.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-30T12:43:28.854 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:28.946 INFO:teuthology.orchestra.run.smithi008.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-30T12:43:29.021 INFO:teuthology.orchestra.run.smithi008.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-30T12:43:29.032 INFO:teuthology.orchestra.run.smithi008.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-30T12:43:29.033 INFO:teuthology.orchestra.run.smithi008.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-30T12:43:29.034 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-30T12:43:29.036 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-30T12:43:29.037 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-30T12:43:29.039 INFO:teuthology.orchestra.run.smithi008.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-30T12:43:29.040 INFO:teuthology.orchestra.run.smithi008.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-30T12:43:29.041 INFO:teuthology.orchestra.run.smithi008.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-30T12:43:29.044 INFO:teuthology.orchestra.run.smithi008.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-30T12:43:29.061 INFO:teuthology.orchestra.run.smithi008.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-30T12:43:29.386 INFO:teuthology.orchestra.run.smithi008.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-30T12:43:29.388 INFO:teuthology.orchestra.run.smithi008.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-30T12:43:29.388 INFO:teuthology.orchestra.run.smithi008.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-30T12:43:29.391 INFO:teuthology.orchestra.run.smithi008.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-30T12:43:29.391 INFO:teuthology.orchestra.run.smithi008.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-30T12:43:29.396 INFO:teuthology.orchestra.run.smithi008.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-30T12:43:29.397 INFO:teuthology.orchestra.run.smithi008.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-30T12:43:29.405 INFO:teuthology.orchestra.run.smithi008.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-30T12:43:29.405 INFO:teuthology.orchestra.run.smithi008.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-30T12:43:29.427 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-common. 2023-10-30T12:43:29.432 INFO:teuthology.orchestra.run.smithi008.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-30T12:43:29.437 INFO:teuthology.orchestra.run.smithi008.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-30T12:43:29.449 INFO:teuthology.orchestra.run.smithi008.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-30T12:43:29.449 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-30T12:43:29.481 INFO:teuthology.orchestra.run.smithi008.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-30T12:43:29.485 INFO:teuthology.orchestra.run.smithi008.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-30T12:43:29.485 INFO:teuthology.orchestra.run.smithi008.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-30T12:43:29.486 INFO:teuthology.orchestra.run.smithi008.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-30T12:43:29.487 INFO:teuthology.orchestra.run.smithi008.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-30T12:43:29.499 INFO:teuthology.orchestra.run.smithi008.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-30T12:43:29.503 INFO:teuthology.orchestra.run.smithi046.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-30T12:43:29.513 INFO:teuthology.orchestra.run.smithi008.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-30T12:43:29.525 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-common (238) ... 2023-10-30T12:43:29.734 INFO:teuthology.orchestra.run.smithi008.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-30T12:43:29.734 INFO:teuthology.orchestra.run.smithi008.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-30T12:43:29.735 INFO:teuthology.orchestra.run.smithi008.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-30T12:43:29.735 INFO:teuthology.orchestra.run.smithi008.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-30T12:43:29.735 INFO:teuthology.orchestra.run.smithi008.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-30T12:43:29.736 INFO:teuthology.orchestra.run.smithi008.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-30T12:43:29.749 INFO:teuthology.orchestra.run.smithi008.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-30T12:43:29.750 INFO:teuthology.orchestra.run.smithi008.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-30T12:43:29.951 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-14. 2023-10-30T12:43:29.964 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:29.994 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:30.145 INFO:teuthology.orchestra.run.smithi008.stdout:Preconfiguring packages ... 2023-10-30T12:43:30.296 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 54.6 MB in 9s (6,008 kB/s) 2023-10-30T12:43:30.437 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package flex. 2023-10-30T12:43:30.512 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-30T12:43:30.516 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:30.552 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-30T12:43:31.207 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:43:31.287 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql. 2023-10-30T12:43:31.310 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-30T12:43:31.345 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql (14+238) ... 2023-10-30T12:43:31.384 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-30T12:43:31.589 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client. 2023-10-30T12:43:31.602 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-30T12:43:31.630 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client (14+238) ... 2023-10-30T12:43:31.775 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:31.891 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-30T12:43:31.904 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-30T12:43:31.941 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-30T12:43:32.260 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl8.6. 2023-10-30T12:43:32.283 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-30T12:43:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:43:32.343 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package bison. 2023-10-30T12:43:32.418 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-30T12:43:32.422 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-30T12:43:32.471 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:43:32.605 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl. 2023-10-30T12:43:32.627 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-30T12:43:32.676 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-30T12:43:32.821 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-30T12:43:32.836 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-30T12:43:32.880 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:43:33.058 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2023-10-30T12:43:33.080 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-30T12:43:33.124 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:43:33.199 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package dump. 2023-10-30T12:43:33.214 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-30T12:43:33.258 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking dump (0.4b47-3) ... 2023-10-30T12:43:33.479 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-30T12:43:33.492 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-30T12:43:33.528 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-30T12:43:33.570 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package indent. 2023-10-30T12:43:33.594 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:33.636 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:43:33.873 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-30T12:43:33.896 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-30T12:43:33.932 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-30T12:43:33.948 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-30T12:43:33.957 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:43:33.991 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:43:34.320 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-30T12:43:34.336 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-30T12:43:34.378 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:43:34.584 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-common (238) ... 2023-10-30T12:43:34.664 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-30T12:43:34.680 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-30T12:43:34.722 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:43:34.755 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-30T12:43:34.881 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:43:35.007 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:35.034 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-30T12:43:35.050 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:35.092 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:35.133 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:43:35.259 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:43:35.353 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-30T12:43:35.369 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-30T12:43:35.385 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:35.411 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:35.697 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjson-perl. 2023-10-30T12:43:35.713 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-30T12:43:35.756 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-30T12:43:35.914 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-30T12:43:36.007 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:43:36.059 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-30T12:43:36.075 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-30T12:43:36.109 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:43:36.393 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:43:36.393 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:43:36.951 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:43:37.077 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:37.178 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-30T12:43:37.279 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:43:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:37.498 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:43:37.553 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-30T12:43:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:43:37.612 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:37.632 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:43:37.872 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-30T12:43:37.889 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-30T12:43:37.930 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:38.246 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:43:38.271 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-30T12:43:38.283 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:43:38.325 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:38.364 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b47-3) ... 2023-10-30T12:43:38.516 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-30T12:43:38.583 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:38.648 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-30T12:43:38.661 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-30T12:43:38.703 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:43:38.733 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:43:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-30T12:43:38.985 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-30T12:43:39.014 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-30T12:43:39.030 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:43:39.072 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:43:39.111 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:43:39.209 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-30T12:43:39.263 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:43:39.364 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:43:39.459 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-30T12:43:39.475 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:39.517 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:39.540 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:43:39.666 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:39.792 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-30T12:43:39.828 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-30T12:43:39.844 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:39.878 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:39.960 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client (14+238) ... 2023-10-30T12:43:40.086 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:43:40.172 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-30T12:43:40.188 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:40.230 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:40.508 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-30T12:43:40.524 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-30T12:43:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-30T12:43:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2023-10-30T12:43:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2023-10-30T12:43:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-30T12:43:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2023-10-30T12:43:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:40.574 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:40.689 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-30T12:43:41.012 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-30T12:43:41.053 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-30T12:43:41.069 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-30T12:43:41.111 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:43:41.398 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-30T12:43:41.464 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package modemmanager. 2023-10-30T12:43:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-30T12:43:41.522 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:41.939 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:42.129 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package network-manager. 2023-10-30T12:43:42.145 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-30T12:43:42.196 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:43:42.208 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-30T12:43:42.334 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-common (238) ... 2023-10-30T12:43:42.819 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ppp. 2023-10-30T12:43:42.836 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-30T12:43:43.176 INFO:teuthology.orchestra.run.smithi046.stdout:Adding user postgres to group ssl-cert 2023-10-30T12:43:43.181 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:43:43.351 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:43.352 INFO:teuthology.orchestra.run.smithi046.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-30T12:43:43.499 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package pptp-linux. 2023-10-30T12:43:43.514 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-30T12:43:43.545 INFO:teuthology.orchestra.run.smithi046.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-30T12:43:43.545 INFO:teuthology.orchestra.run.smithi046.stdout:Removing obsolete dictionary files: 2023-10-30T12:43:43.557 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-30T12:43:43.818 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-30T12:43:43.833 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-30T12:43:43.876 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-30T12:43:44.138 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-30T12:43:44.221 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-30T12:43:44.237 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-30T12:43:44.271 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-client-common (238) ... 2023-10-30T12:43:44.561 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:44.575 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-30T12:43:44.591 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:44.633 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:44.897 INFO:teuthology.orchestra.run.smithi046.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-30T12:43:44.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-30T12:43:44.968 INFO:teuthology.orchestra.run.smithi046.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-30T12:43:44.968 INFO:teuthology.orchestra.run.smithi046.stdout:This user must also own the server process. 2023-10-30T12:43:44.968 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout:The database cluster will be initialized with locale "en_US". 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout:The default text search configuration will be set to "english". 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout:Data page checksums are disabled. 2023-10-30T12:43:44.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:43:44.970 INFO:teuthology.orchestra.run.smithi046.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-30T12:43:44.970 INFO:teuthology.orchestra.run.smithi046.stdout:creating subdirectories ... ok 2023-10-30T12:43:44.970 INFO:teuthology.orchestra.run.smithi046.stdout:selecting dynamic shared memory implementation ... posix 2023-10-30T12:43:44.984 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default max_connections ... 100 2023-10-30T12:43:45.006 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default shared_buffers ... 128MB 2023-10-30T12:43:45.016 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default time zone ... Etc/UTC 2023-10-30T12:43:45.017 INFO:teuthology.orchestra.run.smithi046.stdout:creating configuration files ... ok 2023-10-30T12:43:45.136 INFO:teuthology.orchestra.run.smithi046.stdout:running bootstrap script ... ok 2023-10-30T12:43:45.279 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-common. 2023-10-30T12:43:45.296 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-30T12:43:45.357 INFO:teuthology.orchestra.run.smithi008.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-30T12:43:45.387 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-common (238) ... 2023-10-30T12:43:45.812 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-14. 2023-10-30T12:43:45.828 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-30T12:43:45.858 INFO:teuthology.orchestra.run.smithi046.stdout:performing post-bootstrap initialization ... ok 2023-10-30T12:43:45.874 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:46.598 INFO:teuthology.orchestra.run.smithi046.stdout:syncing data to disk ... ok 2023-10-30T12:43:47.009 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-30T12:43:47.098 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql. 2023-10-30T12:43:47.114 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-30T12:43:47.156 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql (14+238) ... 2023-10-30T12:43:47.377 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-client. 2023-10-30T12:43:47.391 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-30T12:43:47.451 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-client (14+238) ... 2023-10-30T12:43:47.695 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-30T12:43:47.710 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-30T12:43:47.753 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-30T12:43:48.072 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package tcl8.6. 2023-10-30T12:43:48.089 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-30T12:43:48.130 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:43:48.400 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package tcl. 2023-10-30T12:43:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-30T12:43:48.463 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-30T12:43:48.811 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package wpasupplicant. 2023-10-30T12:43:48.827 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-30T12:43:48.870 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:43:49.232 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-30T12:43:49.248 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-30T12:43:49.290 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-30T12:43:49.669 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-30T12:43:49.685 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-30T12:43:49.752 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:43:49.926 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:50.029 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-client-common (238) ... 2023-10-30T12:43:50.060 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-30T12:43:50.186 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql (14+238) ... 2023-10-30T12:43:50.241 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-30T12:43:50.312 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:43:50.367 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-30T12:43:50.438 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:50.502 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:50.611 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-30T12:43:50.700 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:43:50.700 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:43:51.052 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-30T12:43:51.316 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-30T12:43:51.522 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:43:51.635 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:43:51.786 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:43:52.560 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-30T12:43:52.661 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-30T12:43:53.040 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:43:53.040 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-30T12:43:53.164 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-30T12:43:53.273 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-30T12:43:53.730 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-30T12:43:53.856 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-30T12:43:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:43:53.918 DEBUG:teuthology.parallel:result is None 2023-10-30T12:43:53.982 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-30T12:43:54.167 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-30T12:43:54.293 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:54.419 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-30T12:43:54.562 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-30T12:43:55.217 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-30T12:43:55.344 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up dump (0.4b47-3) ... 2023-10-30T12:43:55.439 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-30T12:43:55.503 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:43:55.638 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-30T12:43:55.755 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-30T12:43:55.881 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-30T12:43:56.007 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-30T12:43:56.097 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-30T12:43:56.167 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-30T12:43:56.293 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-30T12:43:56.470 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-30T12:43:56.587 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:56.688 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-30T12:43:56.881 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-client (14+238) ... 2023-10-30T12:43:57.007 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-30T12:43:57.576 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:43:57.576 INFO:teuthology.orchestra.run.smithi008.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-30T12:43:57.576 INFO:teuthology.orchestra.run.smithi008.stdout:which means they are currently configured by ifupdown: 2023-10-30T12:43:57.576 INFO:teuthology.orchestra.run.smithi008.stdout:- ens1f0 2023-10-30T12:43:57.577 INFO:teuthology.orchestra.run.smithi008.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-30T12:43:57.577 INFO:teuthology.orchestra.run.smithi008.stdout:remove their configuration from /etc/network/interfaces. 2023-10-30T12:43:57.577 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:43:57.717 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-30T12:43:58.125 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-30T12:43:58.542 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-30T12:43:59.302 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-30T12:43:59.437 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-30T12:43:59.554 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-30T12:43:59.656 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-common (238) ... 2023-10-30T12:44:00.546 INFO:teuthology.orchestra.run.smithi008.stdout:Adding user postgres to group ssl-cert 2023-10-30T12:44:00.800 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:00.800 INFO:teuthology.orchestra.run.smithi008.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-30T12:44:01.065 INFO:teuthology.orchestra.run.smithi008.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-30T12:44:01.066 INFO:teuthology.orchestra.run.smithi008.stdout:Removing obsolete dictionary files: 2023-10-30T12:44:01.787 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-30T12:44:02.224 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-30T12:44:02.572 INFO:teuthology.orchestra.run.smithi008.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-30T12:44:02.597 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-30T12:44:02.649 INFO:teuthology.orchestra.run.smithi008.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-30T12:44:02.650 INFO:teuthology.orchestra.run.smithi008.stdout:This user must also own the server process. 2023-10-30T12:44:02.650 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:02.650 INFO:teuthology.orchestra.run.smithi008.stdout:The database cluster will be initialized with locale "en_US". 2023-10-30T12:44:02.650 INFO:teuthology.orchestra.run.smithi008.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-30T12:44:02.650 INFO:teuthology.orchestra.run.smithi008.stdout:The default text search configuration will be set to "english". 2023-10-30T12:44:02.651 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:02.651 INFO:teuthology.orchestra.run.smithi008.stdout:Data page checksums are disabled. 2023-10-30T12:44:02.651 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:02.651 INFO:teuthology.orchestra.run.smithi008.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-30T12:44:02.652 INFO:teuthology.orchestra.run.smithi008.stdout:creating subdirectories ... ok 2023-10-30T12:44:02.652 INFO:teuthology.orchestra.run.smithi008.stdout:selecting dynamic shared memory implementation ... posix 2023-10-30T12:44:02.675 INFO:teuthology.orchestra.run.smithi008.stdout:selecting default max_connections ... 100 2023-10-30T12:44:02.712 INFO:teuthology.orchestra.run.smithi008.stdout:selecting default shared_buffers ... 128MB 2023-10-30T12:44:02.726 INFO:teuthology.orchestra.run.smithi008.stdout:selecting default time zone ... Etc/UTC 2023-10-30T12:44:02.727 INFO:teuthology.orchestra.run.smithi008.stdout:creating configuration files ... ok 2023-10-30T12:44:02.904 INFO:teuthology.orchestra.run.smithi008.stdout:running bootstrap script ... ok 2023-10-30T12:44:03.584 INFO:teuthology.orchestra.run.smithi008.stdout:performing post-bootstrap initialization ... ok 2023-10-30T12:44:04.199 INFO:teuthology.orchestra.run.smithi008.stdout:syncing data to disk ... ok 2023-10-30T12:44:04.760 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-30T12:44:08.015 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:44:08.140 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-30T12:44:08.266 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up postgresql (14+238) ... 2023-10-30T12:44:08.401 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-30T12:44:08.510 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-30T12:44:08.782 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:44:08.782 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-30T12:44:09.596 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-30T12:44:09.890 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-30T12:44:10.041 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-30T12:44:11.486 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-30T12:44:11.586 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-30T12:44:12.308 INFO:teuthology.orchestra.run.smithi008.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-30T12:44:12.313 DEBUG:teuthology.parallel:result is None 2023-10-30T12:44:12.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:44:12.543 DEBUG:teuthology.orchestra.run.smithi008:> dpkg-query -W -f '${Version}' ceph 2023-10-30T12:44:12.567 INFO:teuthology.orchestra.run.smithi008.stdout:18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:44:12.567 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:44:12.568 INFO:teuthology.task.install:The correct ceph version 18.0.0-6835-g00c1c13c-1jammy is installed. 2023-10-30T12:44:12.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:44:12.791 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2023-10-30T12:44:12.813 INFO:teuthology.orchestra.run.smithi046.stdout:18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:44:12.813 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6835-g00c1c13c-1jammy 2023-10-30T12:44:12.814 INFO:teuthology.task.install:The correct ceph version 18.0.0-6835-g00c1c13c-1jammy is installed. 2023-10-30T12:44:12.814 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-30T12:44:12.814 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:12.815 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T12:44:12.837 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:12.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T12:44:12.871 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-30T12:44:12.871 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:12.871 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T12:44:12.900 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T12:44:12.964 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:12.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T12:44:12.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T12:44:13.037 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-30T12:44:13.038 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:13.038 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T12:44:13.060 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T12:44:13.124 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:13.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T12:44:13.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T12:44:13.193 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-30T12:44:13.194 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:13.194 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T12:44:13.215 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T12:44:13.276 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:13.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T12:44:13.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T12:44:13.345 INFO:teuthology.run_tasks:Running task ceph... 2023-10-30T12:44:13.411 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-30T12:44:13.411 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /var/log/ceph 2023-10-30T12:44:13.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2023-10-30T12:44:13.433 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-30T12:44:13.433 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T12:44:13.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T12:44:13.497 INFO:tasks.ceph:Creating extra log directories... 2023-10-30T12:44:13.498 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T12:44:13.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T12:44:13.568 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-30T12:44:13.568 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-30T12:44:13.568 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-30T12:44:13.569 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:13.569 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-30T12:44:13.612 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:13.706 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:13.707 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-30T12:44:13.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:13.792 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-30T12:44:13.792 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-30T12:44:13.793 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032', 'branch': 'wip-lusov-balance-after-stop', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7441032', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-lusov-balance-after-stop'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-balance-after-stop', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8397, 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'sleep_before_teardown': 0, 'subset': '464/1024', 'suite': 'fs', 'suite_branch': 'wip-lusov-balance-after-stop', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi008.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiIcfgwrGC9+v3RXe8vNJeksoOyZsreQ1gqopuMNh7A6b/pQ2xJyT5FrS1lYTxOOiBa46uoctB44j3nvKR85mw=', 'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrOaSl/1vsdItzZTYKMwfRP6iubhYHTiH89lRxHjBRZoyL5v6NbgZ2wXiui+zGvgcvVlPNuVTJ+jWYb+TFCaoA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '00c1c13cea21a686b046584596fbeac5315a7d71', 'cluster': 'ceph'}}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-10-30_06:34:16', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-10-30T12:44:13.794 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T12:44:13.797 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T12:44:13.834 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T12:44:13.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T12:44:13.890 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:13.890 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T12:44:13.920 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T12:44:13.921 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2023-10-30T12:44:13.973 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T12:44:13.973 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:13.973 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 901 Links: 1 2023-10-30T12:44:13.973 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:13.973 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-30 12:37:40.960138428 +0000 2023-10-30T12:44:13.974 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-30 12:37:40.584141786 +0000 2023-10-30T12:44:13.974 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-30 12:37:40.584141786 +0000 2023-10-30T12:44:13.974 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-10-30 12:37:40.584141786 +0000 2023-10-30T12:44:13.974 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T12:44:14.035 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-30T12:44:14.035 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-30T12:44:14.035 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000260215 s, 2.0 MB/s 2023-10-30T12:44:14.037 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T12:44:14.090 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2023-10-30T12:44:14.141 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T12:44:14.141 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.141 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 918 Links: 1 2023-10-30T12:44:14.141 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.141 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-30 12:37:41.604132675 +0000 2023-10-30T12:44:14.142 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-30 12:37:41.224136070 +0000 2023-10-30T12:44:14.142 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-30 12:37:41.224136070 +0000 2023-10-30T12:44:14.142 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-10-30 12:37:41.224136070 +0000 2023-10-30T12:44:14.142 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T12:44:14.204 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-30T12:44:14.204 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-30T12:44:14.204 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000266014 s, 1.9 MB/s 2023-10-30T12:44:14.206 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T12:44:14.263 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2023-10-30T12:44:14.313 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T12:44:14.313 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.313 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 935 Links: 1 2023-10-30T12:44:14.314 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.314 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-30 12:37:42.252126887 +0000 2023-10-30T12:44:14.314 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-30 12:37:41.852130460 +0000 2023-10-30T12:44:14.314 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-30 12:37:41.852130460 +0000 2023-10-30T12:44:14.315 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-10-30 12:37:41.852130460 +0000 2023-10-30T12:44:14.315 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T12:44:14.380 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-30T12:44:14.380 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-30T12:44:14.381 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000266651 s, 1.9 MB/s 2023-10-30T12:44:14.382 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T12:44:14.435 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2023-10-30T12:44:14.485 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T12:44:14.485 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.485 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 952 Links: 1 2023-10-30T12:44:14.485 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.486 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-30 12:37:42.912120992 +0000 2023-10-30T12:44:14.486 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-30 12:37:42.520124494 +0000 2023-10-30T12:44:14.486 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-30 12:37:42.520124494 +0000 2023-10-30T12:44:14.486 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-10-30 12:37:42.520124494 +0000 2023-10-30T12:44:14.486 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T12:44:14.548 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-30T12:44:14.548 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-30T12:44:14.548 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000265396 s, 1.9 MB/s 2023-10-30T12:44:14.550 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T12:44:14.607 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:14.607 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:14.607 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T12:44:14.614 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T12:44:14.614 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2023-10-30T12:44:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T12:44:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.666 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 893 Links: 1 2023-10-30T12:44:14.667 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.667 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-30 12:37:06.038888538 +0000 2023-10-30T12:44:14.667 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-30 12:37:05.646899451 +0000 2023-10-30T12:44:14.667 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-30 12:37:05.646899451 +0000 2023-10-30T12:44:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-30T12:44:14.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T12:44:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-30T12:44:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-30T12:44:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000348518 s, 1.5 MB/s 2023-10-30T12:44:14.726 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T12:44:14.775 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2023-10-30T12:44:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T12:44:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.823 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 906 Links: 1 2023-10-30T12:44:14.823 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.823 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-30 12:37:06.618872917 +0000 2023-10-30T12:44:14.824 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-30 12:37:06.254882721 +0000 2023-10-30T12:44:14.824 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-30 12:37:06.254882721 +0000 2023-10-30T12:44:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-30T12:44:14.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T12:44:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-30T12:44:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-30T12:44:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000225013 s, 2.3 MB/s 2023-10-30T12:44:14.881 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T12:44:14.943 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2023-10-30T12:44:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T12:44:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:14.991 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 922 Links: 1 2023-10-30T12:44:14.991 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:14.991 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-30 12:37:07.190857699 +0000 2023-10-30T12:44:14.992 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-30 12:37:06.830867208 +0000 2023-10-30T12:44:14.992 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-30 12:37:06.830867208 +0000 2023-10-30T12:44:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-30T12:44:14.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T12:44:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-30T12:44:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-30T12:44:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000332934 s, 1.5 MB/s 2023-10-30T12:44:15.046 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T12:44:15.097 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2023-10-30T12:44:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T12:44:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T12:44:15.146 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 941 Links: 1 2023-10-30T12:44:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:44:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-30 12:37:07.746843281 +0000 2023-10-30T12:44:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-30 12:37:07.386852616 +0000 2023-10-30T12:44:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-30 12:37:07.386852616 +0000 2023-10-30T12:44:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-30T12:44:15.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T12:44:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-30T12:44:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-30T12:44:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000290879 s, 1.8 MB/s 2023-10-30T12:44:15.202 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T12:44:15.251 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:15.251 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T12:44:15.251 INFO:tasks.ceph:Generating config... 2023-10-30T12:44:15.252 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-30T12:44:15.253 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2023-10-30T12:44:15.253 INFO:tasks.ceph:[client] debug client = 20 2023-10-30T12:44:15.253 INFO:tasks.ceph:[client] debug ms = 1 2023-10-30T12:44:15.254 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-30T12:44:15.254 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-30T12:44:15.254 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-30T12:44:15.254 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-30T12:44:15.255 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-30T12:44:15.255 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-30T12:44:15.255 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-10-30T12:44:15.255 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-10-30T12:44:15.256 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-10-30T12:44:15.256 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-10-30T12:44:15.256 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-30T12:44:15.256 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-30T12:44:15.257 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2023-10-30T12:44:15.257 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-30T12:44:15.257 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-30T12:44:15.257 INFO:tasks.ceph:[mds] mds_dir_prefetch = False 2023-10-30T12:44:15.258 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-10-30T12:44:15.258 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2023-10-30T12:44:15.258 INFO:tasks.ceph:[mds] ms inject delay max = 1 2023-10-30T12:44:15.259 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2023-10-30T12:44:15.259 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2023-10-30T12:44:15.259 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2023-10-30T12:44:15.259 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-30T12:44:15.260 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-30T12:44:15.260 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-30T12:44:15.260 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-30T12:44:15.260 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-30T12:44:15.261 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-30T12:44:15.261 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-30T12:44:15.261 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-30T12:44:15.261 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-30T12:44:15.261 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-30T12:44:15.262 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-30T12:44:15.262 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-30T12:44:15.262 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-30T12:44:15.263 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-30T12:44:15.263 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-30T12:44:15.263 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-30T12:44:15.263 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-30T12:44:15.264 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-30T12:44:15.264 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-30T12:44:15.264 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2023-10-30T12:44:15.264 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-30T12:44:15.265 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-30T12:44:15.265 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-30T12:44:15.265 INFO:tasks.ceph:[osd] ms inject delay max = 1 2023-10-30T12:44:15.265 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2023-10-30T12:44:15.266 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2023-10-30T12:44:15.266 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2023-10-30T12:44:15.266 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-10-30T12:44:15.266 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-30T12:44:15.267 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-10-30T12:44:15.267 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-30T12:44:15.267 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-30T12:44:15.267 INFO:tasks.ceph:Setting up mon.a... 2023-10-30T12:44:15.268 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-30T12:44:15.319 INFO:teuthology.orchestra.run.smithi008.stdout:creating /etc/ceph/ceph.keyring 2023-10-30T12:44:15.332 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-30T12:44:15.387 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:44:15.458 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.8'), ('mon.b', '172.21.15.46'), ('mon.c', '[v2:172.21.15.46:3301,v1:172.21.15.46:6790]')] 2023-10-30T12:44:15.460 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.8,172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-30T12:44:15.462 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:15.462 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T12:44:15.515 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.8 --add b 172.21.15.46 --addv c '[v2:172.21.15.46:3301,v1:172.21.15.46:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:15.607 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:15.607 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: generated fsid d8d9e729-1294-467e-ac5d-e55aafd29b99 2023-10-30T12:44:15.608 INFO:teuthology.orchestra.run.smithi008.stdout:setting min_mon_release = pacific 2023-10-30T12:44:15.608 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 0 2023-10-30T12:44:15.608 INFO:teuthology.orchestra.run.smithi008.stdout:fsid d8d9e729-1294-467e-ac5d-e55aafd29b99 2023-10-30T12:44:15.609 INFO:teuthology.orchestra.run.smithi008.stdout:last_changed 2023-10-30T12:44:15.608341+0000 2023-10-30T12:44:15.609 INFO:teuthology.orchestra.run.smithi008.stdout:created 2023-10-30T12:44:15.608341+0000 2023-10-30T12:44:15.609 INFO:teuthology.orchestra.run.smithi008.stdout:min_mon_release 16 (pacific) 2023-10-30T12:44:15.610 INFO:teuthology.orchestra.run.smithi008.stdout:election_strategy: 1 2023-10-30T12:44:15.610 INFO:teuthology.orchestra.run.smithi008.stdout:0: [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon.a 2023-10-30T12:44:15.610 INFO:teuthology.orchestra.run.smithi008.stdout:1: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2023-10-30T12:44:15.611 INFO:teuthology.orchestra.run.smithi008.stdout:2: [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] mon.c 2023-10-30T12:44:15.611 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-30T12:44:15.613 DEBUG:teuthology.orchestra.run.smithi008:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T12:44:15.623 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d8d9e729-1294-467e-ac5d-e55aafd29b99... 2023-10-30T12:44:15.625 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T12:44:15.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T12:44:15.700 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2023-10-30T12:44:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2023-10-30T12:44:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T12:44:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2023-10-30T12:44:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2023-10-30T12:44:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2023-10-30T12:44:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2023-10-30T12:44:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2023-10-30T12:44:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2023-10-30T12:44:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2023-10-30T12:44:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T12:44:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2023-10-30T12:44:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T12:44:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T12:44:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2023-10-30T12:44:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2023-10-30T12:44:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2023-10-30T12:44:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2023-10-30T12:44:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2023-10-30T12:44:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2023-10-30T12:44:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2023-10-30T12:44:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2023-10-30T12:44:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T12:44:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T12:44:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2023-10-30T12:44:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T12:44:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2023-10-30T12:44:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2023-10-30T12:44:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2023-10-30T12:44:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2023-10-30T12:44:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2023-10-30T12:44:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.8,172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790]" 2023-10-30T12:44:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2023-10-30T12:44:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = d8d9e729-1294-467e-ac5d-e55aafd29b99 2023-10-30T12:44:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.717 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2023-10-30T12:44:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2023-10-30T12:44:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2023-10-30T12:44:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2023-10-30T12:44:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2023-10-30T12:44:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2023-10-30T12:44:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2023-10-30T12:44:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2023-10-30T12:44:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2023-10-30T12:44:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2023-10-30T12:44:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2023-10-30T12:44:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2023-10-30T12:44:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2023-10-30T12:44:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2023-10-30T12:44:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2023-10-30T12:44:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2023-10-30T12:44:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2023-10-30T12:44:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2023-10-30T12:44:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2023-10-30T12:44:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2023-10-30T12:44:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2023-10-30T12:44:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2023-10-30T12:44:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2023-10-30T12:44:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2023-10-30T12:44:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2023-10-30T12:44:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2023-10-30T12:44:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-30T12:44:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2023-10-30T12:44:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2023-10-30T12:44:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: mon client directed command retry = 5 2023-10-30T12:44:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T12:44:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2023-10-30T12:44:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2023-10-30T12:44:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay max = 1 2023-10-30T12:44:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay probability = 0.005 2023-10-30T12:44:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay type = client mds 2023-10-30T12:44:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject socket failures = 2500 2023-10-30T12:44:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: osd blocked scrub grace period = 3600 2023-10-30T12:44:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2023-10-30T12:44:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: osd max markdown count = 1000 2023-10-30T12:44:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2023-10-30T12:44:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-10-30T12:44:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.737 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2023-10-30T12:44:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-30T12:44:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2023-10-30T12:44:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-10-30T12:44:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-10-30T12:44:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:44:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:44:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2023-10-30T12:44:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.740 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2023-10-30T12:44:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-30T12:44:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-10-30T12:44:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2023-10-30T12:44:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-10-30T12:44:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2023-10-30T12:44:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2023-10-30T12:44:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:44:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2023-10-30T12:44:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2023-10-30T12:44:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:44:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T12:44:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T12:44:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T12:44:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T12:44:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T12:44:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T12:44:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2023-10-30T12:44:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2023-10-30T12:44:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2023-10-30T12:44:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2023-10-30T12:44:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2023-10-30T12:44:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T12:44:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T12:44:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2023-10-30T12:44:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: client_shutdown_timeout = 120 2023-10-30T12:44:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2023-10-30T12:44:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-30T12:44:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-10-30T12:44:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-10-30T12:44:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e] 2023-10-30T12:44:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-10-30T12:44:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-10-30T12:44:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f] 2023-10-30T12:44:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2023-10-30T12:44:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2023-10-30T12:44:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2023-10-30T12:44:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-30T12:44:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: mds bal fragment size max = 10000 2023-10-30T12:44:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: mds bal merge size = 5 2023-10-30T12:44:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: mds bal split bits = 3 2023-10-30T12:44:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: mds bal split size = 100 2023-10-30T12:44:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2023-10-30T12:44:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2023-10-30T12:44:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: mds oft prefetch dirfrags = True 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: mds_dir_prefetch = False 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: mds_sleep_rank_change = 5000000.0 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: mon client directed command retry = 5 2023-10-30T12:44:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay max = 1 2023-10-30T12:44:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay probability = 0.005 2023-10-30T12:44:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject delay type = client mds osd 2023-10-30T12:44:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject socket failures = 2500 2023-10-30T12:44:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-10-30T12:44:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-10-30T12:44:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-10-30T12:44:15.758 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2023-10-30T12:44:15.758 INFO:teuthology.orchestra.run.smithi008.stdout: chdir = "" 2023-10-30T12:44:15.759 INFO:teuthology.orchestra.run.smithi008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T12:44:15.759 INFO:teuthology.orchestra.run.smithi008.stdout: auth supported = cephx 2023-10-30T12:44:15.759 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.759 INFO:teuthology.orchestra.run.smithi008.stdout: filestore xattr use omap = true 2023-10-30T12:44:15.759 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.760 INFO:teuthology.orchestra.run.smithi008.stdout: mon clock drift allowed = 1.000 2023-10-30T12:44:15.760 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.760 INFO:teuthology.orchestra.run.smithi008.stdout: osd crush chooseleaf type = 0 2023-10-30T12:44:15.760 INFO:teuthology.orchestra.run.smithi008.stdout: auth debug = true 2023-10-30T12:44:15.760 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on old message = true 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on bug = true 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: mon pg warn max object skew = 0 2023-10-30T12:44:15.761 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.762 INFO:teuthology.orchestra.run.smithi008.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T12:44:15.762 INFO:teuthology.orchestra.run.smithi008.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T12:44:15.762 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.762 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default size = 2 2023-10-30T12:44:15.762 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.763 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow primary affinity = true 2023-10-30T12:44:15.763 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow pg remap = true 2023-10-30T12:44:15.763 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on legacy crush tunables = false 2023-10-30T12:44:15.763 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on crush straw calc version zero = false 2023-10-30T12:44:15.763 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on no sortbitwise = false 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on osd down out interval zero = false 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on too few osds = false 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: mon_allow_pool_size_one = true 2023-10-30T12:44:15.764 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.765 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T12:44:15.765 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.765 INFO:teuthology.orchestra.run.smithi008.stdout: osd default data pool replay window = 5 2023-10-30T12:44:15.765 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.765 INFO:teuthology.orchestra.run.smithi008.stdout: mon allow pool delete = true 2023-10-30T12:44:15.766 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.766 INFO:teuthology.orchestra.run.smithi008.stdout: mon cluster log file level = debug 2023-10-30T12:44:15.766 INFO:teuthology.orchestra.run.smithi008.stdout: debug asserts on shutdown = true 2023-10-30T12:44:15.766 INFO:teuthology.orchestra.run.smithi008.stdout: mon health detail to clog = false 2023-10-30T12:44:15.766 INFO:teuthology.orchestra.run.smithi008.stdout: mon host = "172.21.15.8,172.21.15.46,[v2:172.21.15.46:3301,v1:172.21.15.46:6790]" 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on skipped message = False 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = d8d9e729-1294-467e-ac5d-e55aafd29b99 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout:[osd] 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout: osd journal size = 100 2023-10-30T12:44:15.767 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.768 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub load threshold = 5.0 2023-10-30T12:44:15.768 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub max interval = 600 2023-10-30T12:44:15.768 INFO:teuthology.orchestra.run.smithi008.stdout: osd mclock profile = high_recovery_ops 2023-10-30T12:44:15.768 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.768 INFO:teuthology.orchestra.run.smithi008.stdout: osd recover clone overlap = true 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: osd recovery max chunk = 1048576 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug shutdown = true 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug op order = true 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug verify stray on activate = true 2023-10-30T12:44:15.769 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.770 INFO:teuthology.orchestra.run.smithi008.stdout: osd open classes on start = true 2023-10-30T12:44:15.770 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug pg log writeout = true 2023-10-30T12:44:15.770 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.770 INFO:teuthology.orchestra.run.smithi008.stdout: osd deep scrub update digest min age = 30 2023-10-30T12:44:15.770 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.771 INFO:teuthology.orchestra.run.smithi008.stdout: osd map max advance = 10 2023-10-30T12:44:15.771 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.771 INFO:teuthology.orchestra.run.smithi008.stdout: journal zero on create = true 2023-10-30T12:44:15.771 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.771 INFO:teuthology.orchestra.run.smithi008.stdout: filestore ondisk finisher threads = 3 2023-10-30T12:44:15.772 INFO:teuthology.orchestra.run.smithi008.stdout: filestore apply finisher threads = 3 2023-10-30T12:44:15.772 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.772 INFO:teuthology.orchestra.run.smithi008.stdout: bdev debug aio = true 2023-10-30T12:44:15.772 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug misdirected ops = true 2023-10-30T12:44:15.772 INFO:teuthology.orchestra.run.smithi008.stdout: bdev async discard = True 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: bdev enable discard = True 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore allocator = bitmap 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore block size = 96636764160 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore fsck on mount = True 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluefs = 1/20 2023-10-30T12:44:15.773 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluestore = 1/20 2023-10-30T12:44:15.774 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-30T12:44:15.774 INFO:teuthology.orchestra.run.smithi008.stdout: debug osd = 20 2023-10-30T12:44:15.774 INFO:teuthology.orchestra.run.smithi008.stdout: debug rocksdb = 4/10 2023-10-30T12:44:15.774 INFO:teuthology.orchestra.run.smithi008.stdout: mon client directed command retry = 5 2023-10-30T12:44:15.774 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T12:44:15.775 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd full ratio = 0.9 2023-10-30T12:44:15.775 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd nearfull ratio = 0.8 2023-10-30T12:44:15.775 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay max = 1 2023-10-30T12:44:15.775 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay probability = 0.005 2023-10-30T12:44:15.775 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay type = client mds 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject socket failures = 2500 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: osd blocked scrub grace period = 3600 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: osd failsafe full ratio = 0.95 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: osd max markdown count = 1000 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: osd objectstore = bluestore 2023-10-30T12:44:15.776 INFO:teuthology.orchestra.run.smithi008.stdout: osd op complaint time = 180 2023-10-30T12:44:15.777 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.777 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr] 2023-10-30T12:44:15.777 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-30T12:44:15.777 INFO:teuthology.orchestra.run.smithi008.stdout: debug mgr = 20 2023-10-30T12:44:15.777 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout: mgr/telemetry/nag = false 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.778 INFO:teuthology.orchestra.run.smithi008.stdout:[mon] 2023-10-30T12:44:15.779 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-30T12:44:15.779 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-10-30T12:44:15.779 INFO:teuthology.orchestra.run.smithi008.stdout: debug paxos = 20 2023-10-30T12:44:15.779 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-10-30T12:44:15.779 INFO:teuthology.orchestra.run.smithi008.stdout: mon data avail warn = 5 2023-10-30T12:44:15.780 INFO:teuthology.orchestra.run.smithi008.stdout: mon mgr mkfs grace = 240 2023-10-30T12:44:15.780 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-10-30T12:44:15.780 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd reporter subtree level = osd 2023-10-30T12:44:15.780 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd prime pg temp = true 2023-10-30T12:44:15.780 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-10-30T12:44:15.781 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.781 INFO:teuthology.orchestra.run.smithi008.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T12:44:15.781 INFO:teuthology.orchestra.run.smithi008.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T12:44:15.781 INFO:teuthology.orchestra.run.smithi008.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T12:44:15.781 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: # 1m isn't quite enough 2023-10-30T12:44:15.782 INFO:teuthology.orchestra.run.smithi008.stdout: mon_down_mkfs_grace = 2m 2023-10-30T12:44:15.783 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.783 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_filestore_osds = false 2023-10-30T12:44:15.783 INFO:teuthology.orchestra.run.smithi008.stdout: mon op complaint time = 120 2023-10-30T12:44:15.783 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:15.783 INFO:teuthology.orchestra.run.smithi008.stdout:[client] 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: rgw cache enabled = true 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable ops log = true 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable usage log = true 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T12:44:15.784 INFO:teuthology.orchestra.run.smithi008.stdout: client mount timeout = 600 2023-10-30T12:44:15.785 INFO:teuthology.orchestra.run.smithi008.stdout: client_shutdown_timeout = 120 2023-10-30T12:44:15.785 INFO:teuthology.orchestra.run.smithi008.stdout: debug client = 20 2023-10-30T12:44:15.785 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-30T12:44:15.785 INFO:teuthology.orchestra.run.smithi008.stdout: rados mon op timeout = 900 2023-10-30T12:44:15.785 INFO:teuthology.orchestra.run.smithi008.stdout: rados osd op timeout = 900 2023-10-30T12:44:15.786 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.a] 2023-10-30T12:44:15.786 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.b] 2023-10-30T12:44:15.786 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.c] 2023-10-30T12:44:15.786 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.a] 2023-10-30T12:44:15.787 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.c] 2023-10-30T12:44:15.787 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.e] 2023-10-30T12:44:15.787 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.b] 2023-10-30T12:44:15.787 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.d] 2023-10-30T12:44:15.787 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.f] 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout:[mds] 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout: debug mds = 20 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout: debug mds balancer = 20 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout: mds bal fragment size max = 10000 2023-10-30T12:44:15.788 INFO:teuthology.orchestra.run.smithi008.stdout: mds bal merge size = 5 2023-10-30T12:44:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: mds bal split bits = 3 2023-10-30T12:44:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: mds bal split size = 100 2023-10-30T12:44:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: mds debug frag = True 2023-10-30T12:44:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: mds debug scatterstat = True 2023-10-30T12:44:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: mds oft prefetch dirfrags = True 2023-10-30T12:44:15.790 INFO:teuthology.orchestra.run.smithi008.stdout: mds op complaint time = 180 2023-10-30T12:44:15.790 INFO:teuthology.orchestra.run.smithi008.stdout: mds verify scatter = True 2023-10-30T12:44:15.790 INFO:teuthology.orchestra.run.smithi008.stdout: mds_dir_prefetch = False 2023-10-30T12:44:15.790 INFO:teuthology.orchestra.run.smithi008.stdout: mds_sleep_rank_change = 5000000.0 2023-10-30T12:44:15.790 INFO:teuthology.orchestra.run.smithi008.stdout: mon client directed command retry = 5 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay max = 1 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay probability = 0.005 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject delay type = client mds osd 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject socket failures = 2500 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: osd op complaint time = 180 2023-10-30T12:44:15.791 INFO:teuthology.orchestra.run.smithi008.stdout: rados mon op timeout = 900 2023-10-30T12:44:15.792 INFO:teuthology.orchestra.run.smithi008.stdout: rados osd op timeout = 900 2023-10-30T12:44:15.792 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-30T12:44:15.793 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-30T12:44:15.864 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-30T12:44:15.865 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:15.865 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-30T12:44:15.874 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:15.874 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-30T12:44:15.924 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:44:15.924 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:15.925 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T12:44:15.925 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:44:16.001 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:16.002 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:16.052 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:44:16.053 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:16.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T12:44:16.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T12:44:16.077 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:16.078 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:16.126 INFO:tasks.ceph:Setting up mon nodes... 2023-10-30T12:44:16.126 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-30T12:44:16.127 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T12:44:16.184 INFO:teuthology.orchestra.run.smithi008.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T12:44:16.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T12:44:16.235 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T12:44:16.247 INFO:tasks.ceph:Setting up mds nodes... 2023-10-30T12:44:16.248 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-30T12:44:16.307 INFO:teuthology.orchestra.run.smithi008.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-30T12:44:16.311 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-30T12:44:16.332 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-30T12:44:16.433 INFO:teuthology.orchestra.run.smithi008.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-30T12:44:16.437 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-30T12:44:16.458 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-10-30T12:44:16.556 INFO:teuthology.orchestra.run.smithi008.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-10-30T12:44:16.560 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-10-30T12:44:16.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-30T12:44:16.627 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-30T12:44:16.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-30T12:44:16.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-30T12:44:16.726 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-30T12:44:16.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-30T12:44:16.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-10-30T12:44:16.871 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-10-30T12:44:16.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-10-30T12:44:16.928 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-30T12:44:16.929 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-30T12:44:16.974 INFO:teuthology.orchestra.run.smithi008.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-30T12:44:16.991 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-30T12:44:16.992 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T12:44:16.992 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-30T12:44:17.056 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:17.056 INFO:tasks.ceph:role: osd.0 2023-10-30T12:44:17.057 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:44:17.057 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T12:44:17.120 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:17.120 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:17.120 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:17.120 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:17.120 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:17.121 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:17.121 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:17.121 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:17.121 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:17.121 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:17.144 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-10-30T12:44:17.147 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-10-30T12:44:17.147 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-30T12:44:17.213 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-30T12:44:17.272 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:17.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:17.274 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-30T12:44:17.340 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:17.340 INFO:tasks.ceph:role: osd.1 2023-10-30T12:44:17.341 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:44:17.341 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T12:44:17.404 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:17.404 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:17.404 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:17.404 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:17.405 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:17.428 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-10-30T12:44:17.431 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-10-30T12:44:17.431 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-30T12:44:17.500 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-30T12:44:17.560 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:17.561 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-30T12:44:17.624 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:17.624 INFO:tasks.ceph:role: osd.2 2023-10-30T12:44:17.624 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:44:17.625 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T12:44:17.688 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:17.688 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:17.688 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:17.689 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:17.689 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:17.689 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:17.689 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:17.690 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:17.690 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:17.690 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:17.713 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-10-30T12:44:17.716 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-10-30T12:44:17.716 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-30T12:44:17.789 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-30T12:44:17.848 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:17.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:17.850 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-30T12:44:17.916 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:17.917 INFO:tasks.ceph:role: osd.3 2023-10-30T12:44:17.917 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:44:17.917 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T12:44:17.979 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:17.980 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:17.980 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:17.980 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:17.980 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:17.981 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:17.981 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:17.981 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:17.981 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:17.982 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:18.004 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-10-30T12:44:18.006 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-10-30T12:44:18.006 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-30T12:44:18.078 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-30T12:44:18.136 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:18.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:18.137 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.264+0000 7fbbcb82d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-30T12:44:18.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.264+0000 7fbbcb82d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-30T12:44:18.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.264+0000 7fbbcb82d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:44:18.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.264+0000 7fbbcb82d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:44:18.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.264+0000 7fbbcb82d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T12:44:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:18.268+0000 7fbbcb82d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-30T12:44:21.246 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-30T12:44:21.268 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:21.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-30T12:44:21.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-30T12:44:21.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:44:21.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:44:21.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T12:44:21.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:21.372+0000 7f7b50c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-30T12:44:24.368 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-30T12:44:24.390 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:24.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.488+0000 7f0a3b695640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-30T12:44:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.492+0000 7f0a3b695640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-30T12:44:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.492+0000 7f0a3b695640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:44:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.492+0000 7f0a3b695640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:44:24.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.492+0000 7f0a3b695640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T12:44:24.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:24.492+0000 7f0a3b695640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-30T12:44:27.501 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-30T12:44:27.523 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:27.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-30T12:44:27.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-30T12:44:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:44:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:44:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T12:44:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:27.619+0000 7fc066315640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-30T12:44:30.611 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-30T12:44:30.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-30T12:44:30.647 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:30.648 INFO:tasks.ceph:role: osd.4 2023-10-30T12:44:30.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:44:30.648 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T12:44:30.704 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:30.704 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:30.704 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:30.704 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:30.705 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:30.705 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:30.705 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:30.705 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:30.705 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:30.706 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-10-30T12:44:30.730 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-30T12:44:30.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-30T12:44:30.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-30T12:44:30.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:30.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-30T12:44:30.899 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:30.899 INFO:tasks.ceph:role: osd.5 2023-10-30T12:44:30.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:44:30.900 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T12:44:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:30.953 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:30.955 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-10-30T12:44:30.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-30T12:44:30.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-30T12:44:31.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-30T12:44:31.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:31.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-30T12:44:31.145 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:31.145 INFO:tasks.ceph:role: osd.6 2023-10-30T12:44:31.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:44:31.146 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T12:44:31.201 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:31.202 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:31.202 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:31.203 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:31.203 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:31.225 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-10-30T12:44:31.227 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-30T12:44:31.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-30T12:44:31.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-30T12:44:31.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:31.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-30T12:44:31.391 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T12:44:31.391 INFO:tasks.ceph:role: osd.7 2023-10-30T12:44:31.392 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:44:31.392 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T12:44:31.444 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T12:44:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T12:44:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T12:44:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-30T12:44:31.446 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T12:44:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-30T12:44:31.446 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T12:44:31.446 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-30T12:44:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T12:44:31.447 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T12:44:31.468 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-10-30T12:44:31.470 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-30T12:44:31.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-30T12:44:31.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-30T12:44:31.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:44:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-30T12:44:31.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-30T12:44:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-30T12:44:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:44:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:44:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T12:44:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:31.698+0000 7f470902c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-30T12:44:34.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-30T12:44:34.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-30T12:44:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-30T12:44:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:44:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:44:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T12:44:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:34.794+0000 7f5c6d7d0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-30T12:44:37.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-30T12:44:37.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-30T12:44:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-30T12:44:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:44:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:44:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T12:44:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:37.942+0000 7fa1a84a4640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-30T12:44:40.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-30T12:44:41.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-30T12:44:41.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-30T12:44:41.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:44:41.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:44:41.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T12:44:41.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:41.094+0000 7fcad015d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-30T12:44:43.795 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:43.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:44:44.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-30T12:44:44.118 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-30T12:44:44.119 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.119 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-30T12:44:44.142 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.142 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-30T12:44:44.205 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.206 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-30T12:44:44.269 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-10-30T12:44:44.333 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.333 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-30T12:44:44.397 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.398 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-30T12:44:44.461 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.462 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-30T12:44:44.525 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:44.525 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-30T12:44:44.588 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-30T12:44:44.602 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-30T12:44:44.657 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-30T12:44:44.716 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-10-30T12:44:44.772 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-30T12:44:44.828 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-30T12:44:44.885 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-30T12:44:44.941 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:44.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-30T12:44:45.001 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:45.001 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-30T12:44:45.011 INFO:tasks.ceph:Adding keys to all mons... 2023-10-30T12:44:45.011 DEBUG:teuthology.orchestra.run.smithi008:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T12:44:45.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T12:44:45.067 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2023-10-30T12:44:45.068 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9lZoMTCxAA9L3o/DWYhUo+pkEvft7+SA== 2023-10-30T12:44:45.068 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-10-30T12:44:45.068 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9l35poEhAA5ZrfTXkXYeQCQkd9gPjegQ== 2023-10-30T12:44:45.068 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-10-30T12:44:45.069 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9l5nHmGRAA19fiKXogkzkPt4HX2ltIOw== 2023-10-30T12:44:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e] 2023-10-30T12:44:45.069 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9le4VAIRAAdiJ39KwkYxMtvlQZyGfTFw== 2023-10-30T12:44:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2023-10-30T12:44:45.070 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAipT9lYSMLEBAAhcHOMm9ZiPJ+u//f6dN1kQ== 2023-10-30T12:44:45.070 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2023-10-30T12:44:45.070 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAlpT9lqLxdFhAApETK+McQlqxuN+2Hqnjzsg== 2023-10-30T12:44:45.070 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2023-10-30T12:44:45.070 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAopT9lfSVxHRAAi5XXD9WGIXSQq23c2VgplA== 2023-10-30T12:44:45.071 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2023-10-30T12:44:45.071 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQArpT9l2/gZJRAAbTnxqhP7OoU31o0RhPOXfg== 2023-10-30T12:44:45.071 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2023-10-30T12:44:45.071 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9lJmLhDRAA/iBogPfdox/NeRq8klL6CA== 2023-10-30T12:44:45.072 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-10-30T12:44:45.072 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9lRyhHJRAAU25UoaQZKaih7KDGm51UBw== 2023-10-30T12:44:45.072 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-10-30T12:44:45.072 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9lDDIjKxAAz94kYfzgYwBCG3COBGaJZQ== 2023-10-30T12:44:45.073 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f] 2023-10-30T12:44:45.073 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9l1e/IMxAAfiZJ0Sc8oFAtHIiKJ/IC1w== 2023-10-30T12:44:45.073 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2023-10-30T12:44:45.073 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAvpT9lxjDBKRAAFegrimU55gvR8XigfoC+tg== 2023-10-30T12:44:45.073 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2023-10-30T12:44:45.074 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAypT9lTyd3LxAAr/iaTh3L90elvBSwnkwkDA== 2023-10-30T12:44:45.074 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2023-10-30T12:44:45.074 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA1pT9lrXZCOBAA5Wr3tXp0YzK9I38nC3HE2g== 2023-10-30T12:44:45.074 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2023-10-30T12:44:45.075 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA5pT9l/7KtBRAAh9EDzq5w2XvAHuWv8f1yeA== 2023-10-30T12:44:45.075 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-10-30T12:44:45.075 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAgpT9lPrgsOhAAbwYwKoG0EHvCzLJZ9sIezA== 2023-10-30T12:44:45.077 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr.y] 2023-10-30T12:44:45.077 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9lZoMTCxAA9L3o/DWYhUo+pkEvft7+SA== 2023-10-30T12:44:45.077 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.a] 2023-10-30T12:44:45.077 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9l35poEhAA5ZrfTXkXYeQCQkd9gPjegQ== 2023-10-30T12:44:45.078 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.c] 2023-10-30T12:44:45.078 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9l5nHmGRAA19fiKXogkzkPt4HX2ltIOw== 2023-10-30T12:44:45.078 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.e] 2023-10-30T12:44:45.078 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9le4VAIRAAdiJ39KwkYxMtvlQZyGfTFw== 2023-10-30T12:44:45.079 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.0] 2023-10-30T12:44:45.079 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAipT9lYSMLEBAAhcHOMm9ZiPJ+u//f6dN1kQ== 2023-10-30T12:44:45.079 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.1] 2023-10-30T12:44:45.079 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAlpT9lqLxdFhAApETK+McQlqxuN+2Hqnjzsg== 2023-10-30T12:44:45.079 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.2] 2023-10-30T12:44:45.080 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAopT9lfSVxHRAAi5XXD9WGIXSQq23c2VgplA== 2023-10-30T12:44:45.080 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.3] 2023-10-30T12:44:45.080 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQArpT9l2/gZJRAAbTnxqhP7OoU31o0RhPOXfg== 2023-10-30T12:44:45.080 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr.x] 2023-10-30T12:44:45.081 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9lJmLhDRAA/iBogPfdox/NeRq8klL6CA== 2023-10-30T12:44:45.081 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.b] 2023-10-30T12:44:45.081 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9lRyhHJRAAU25UoaQZKaih7KDGm51UBw== 2023-10-30T12:44:45.081 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.d] 2023-10-30T12:44:45.081 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9lDDIjKxAAz94kYfzgYwBCG3COBGaJZQ== 2023-10-30T12:44:45.082 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.f] 2023-10-30T12:44:45.082 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9l1e/IMxAAfiZJ0Sc8oFAtHIiKJ/IC1w== 2023-10-30T12:44:45.082 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.4] 2023-10-30T12:44:45.082 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAvpT9lxjDBKRAAFegrimU55gvR8XigfoC+tg== 2023-10-30T12:44:45.082 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.5] 2023-10-30T12:44:45.083 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAypT9lTyd3LxAAr/iaTh3L90elvBSwnkwkDA== 2023-10-30T12:44:45.083 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.6] 2023-10-30T12:44:45.083 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQA1pT9lrXZCOBAA5Wr3tXp0YzK9I38nC3HE2g== 2023-10-30T12:44:45.083 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.7] 2023-10-30T12:44:45.084 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQA5pT9l/7KtBRAAh9EDzq5w2XvAHuWv8f1yeA== 2023-10-30T12:44:45.084 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2023-10-30T12:44:45.084 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQAgpT9lPrgsOhAAbwYwKoG0EHvCzLJZ9sIezA== 2023-10-30T12:44:45.084 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:44:45.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:44:45.180 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.243 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.304 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.377 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.433 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.505 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.589 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:45.677 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:44:45.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T12:44:45.770 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.861 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.921 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:45.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-30T12:44:46.004 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.061 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.139 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.222 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T12:44:46.307 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T12:44:46.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T12:44:46.393 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-30T12:44:46.394 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-30T12:44:46.415 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:44:46.530 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-30T12:44:46.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-30T12:44:46.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:44:46.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-30T12:44:46.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-30T12:44:46.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T12:44:46.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-30T12:44:46.839 DEBUG:teuthology.orchestra.run.smithi008:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:46.843 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T12:44:46.884 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-30T12:44:46.885 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T12:44:46.885 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-30T12:44:46.895 INFO:tasks.ceph.mon.a:Started 2023-10-30T12:44:46.895 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T12:44:46.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-30T12:44:46.926 INFO:tasks.ceph.mon.b:Started 2023-10-30T12:44:46.926 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T12:44:46.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-30T12:44:46.929 INFO:tasks.ceph.mon.c:Started 2023-10-30T12:44:46.929 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-30T12:44:46.929 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-30T12:44:46.930 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-30T12:44:46.933 INFO:tasks.ceph.mgr.y:Started 2023-10-30T12:44:46.934 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-30T12:44:46.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-30T12:44:46.936 INFO:tasks.ceph.mgr.x:Started 2023-10-30T12:44:46.937 INFO:tasks.ceph:Setting crush tunables to default 2023-10-30T12:44:46.937 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd crush tunables default 2023-10-30T12:44:47.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 Processor -- start 2023-10-30T12:44:47.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 -- start start 2023-10-30T12:44:47.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:47.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c153f80 con 0x7f034c0c4410 2023-10-30T12:44:47.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c159df0 con 0x7f034c153ba0 2023-10-30T12:44:47.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f03516f7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c1597c0 con 0x7f034c0c4810 2023-10-30T12:44:47.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f0343fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f034c0c4410 0x7f034c0cf490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:49932/0 (socket says 172.21.15.8:49932) 2023-10-30T12:44:47.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 learned_addr learned my addr 172.21.15.8:0/2605695058 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:44:47.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.079+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:47.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.083+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:44:47.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.083+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:47.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.083+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:47.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.083+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:44:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.287+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.287+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:44:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.287+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.287+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.287+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-30T12:44:47.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.691+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:47.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.691+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:44:47.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.691+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:47.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.691+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:47.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:47.691+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-30T12:44:48.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:48.495+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:48.495+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-30T12:44:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:48.495+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:48.495+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:48.495+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-30T12:44:50.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 msgr2=0x7f034c0cb4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:44:50.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f034c0c4810 0x7f034c0cb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:50.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 >> v1:172.21.15.46:6789/0 conn(0x7f034c153ba0 legacy=0x7f034c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:50.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 >> v1:172.21.15.8:6789/0 conn(0x7f034c0c4410 legacy=0x7f034c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:50.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:50.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:50.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c1597c0 con 0x7f032c006200 2023-10-30T12:44:50.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c008d60 con 0x7f032c003740 2023-10-30T12:44:50.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.079+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-30T12:44:50.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.083+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:44:50.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-30T12:44:50.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-30T12:44:50.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-30T12:44:50.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.287+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-30T12:44:50.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:50.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-30T12:44:50.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-30T12:44:50.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-30T12:44:50.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:50.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:50.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:50.691+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-30T12:44:51.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-30T12:44:51.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:51.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:51.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:51.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-30T12:44:51.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-30T12:44:51.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f034affd640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:44:51.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:44:51.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:51.495+0000 7f034affd640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-30T12:44:52.591 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:52.587+0000 7ffbcd847180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T12:44:53.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 msgr2=0x7f032c005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:44:53.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c003740 0x7f032c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 msgr2=0x7f032c0086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:44:53.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c006200 0x7f032c0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f034b7fe640 1 -- 172.21.15.8:0/2605695058 reap_dead start 2023-10-30T12:44:53.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c003b60 0x7f032c006690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 0x7f032c007080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c15d580 con 0x7f034c0c4410 2023-10-30T12:44:53.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c1597c0 con 0x7f032c003b60 2023-10-30T12:44:53.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.079+0000 7f03497fa640 1 -- 172.21.15.8:0/2605695058 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032c008d60 con 0x7f032c006bd0 2023-10-30T12:44:53.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.8:3300/0 conn(0x7f032c003b60 0x7f032c006690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 0x7f032c007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 msgr2=0x7f032c007080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:53.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 0x7f032c007080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 >> v1:172.21.15.8:6789/0 conn(0x7f034c0c4410 legacy=0x7f032c00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 -- 172.21.15.8:0/2605695058 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0087b0 con 0x7f032c003b60 2023-10-30T12:44:53.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034b7fe640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 0x7f032c007080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:53.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034a7fc640 1 -- 172.21.15.8:0/2605695058 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973677412 0 0) 0x7f034c15d580 con 0x7f034c0c4410 2023-10-30T12:44:53.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f0343fff640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f032c003b60 0x7f032c006690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f032c006040 tx=0x7f033401cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1162a198807ac8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034a7fc640 1 -- 172.21.15.8:0/2605695058 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03340a5d80 con 0x7f032c003b60 2023-10-30T12:44:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034a7fc640 1 -- 172.21.15.8:0/2605695058 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03340a9070 con 0x7f032c003b60 2023-10-30T12:44:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.083+0000 7f034a7fc640 1 -- 172.21.15.8:0/2605695058 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03340af640 con 0x7f032c003b60 2023-10-30T12:44:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- 172.21.15.8:0/2605695058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f032c003b60 msgr2=0x7f032c006690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f032c003b60 0x7f032c006690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f032c006040 tx=0x7f033401cfb0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- 172.21.15.8:0/2605695058 shutdown_connections 2023-10-30T12:44:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- 172.21.15.8:0/2605695058 >> v2:172.21.15.46:3300/0 conn(0x7f032c006bd0 0x7f032c007080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- 172.21.15.8:0/2605695058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f032c003b60 0x7f032c006690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- 172.21.15.8:0/2605695058 >> 172.21.15.8:0/2605695058 conn(0x7f034c071790 msgr2=0x7f034c071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:53.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- 172.21.15.8:0/2605695058 shutdown_connections 2023-10-30T12:44:53.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- 172.21.15.8:0/2605695058 wait complete. 2023-10-30T12:44:53.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 Processor -- start 2023-10-30T12:44:53.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- start start 2023-10-30T12:44:53.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c003b60 0x7f034c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c006bd0 0x7f034c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15b9f0 con 0x7f034c0c4410 2023-10-30T12:44:53.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15ab00 con 0x7f032c003b60 2023-10-30T12:44:53.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.087+0000 7f03516f7640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15ade0 con 0x7f032c006bd0 2023-10-30T12:44:53.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55678/0 (socket says 172.21.15.8:55678) 2023-10-30T12:44:53.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 -- 172.21.15.8:0/3334738325 learned_addr learned my addr 172.21.15.8:0/3334738325 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:44:53.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034affd640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c003b60 0x7f034c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f0343fff640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c006bd0 0x7f034c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 -- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c006bd0 msgr2=0x7f034c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c006bd0 0x7f034c15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 -- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c003b60 msgr2=0x7f034c15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c003b60 0x7f034c15be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 -- 172.21.15.8:0/3334738325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033400f040 con 0x7f034c0c4410 2023-10-30T12:44:53.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f034b7fe640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f032c008d00 tx=0x7f0344014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344004720 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034400f040 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c15b0c0 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344017680 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c15b680 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0344017ae0 con 0x7f034c0c4410 2023-10-30T12:44:53.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0344020c90 con 0x7f034c0c4410 2023-10-30T12:44:53.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.091+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c15bb60 con 0x7f034c0c4410 2023-10-30T12:44:53.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.103+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f034401b030 con 0x7f034c0c4410 2023-10-30T12:44:53.225 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:53.223+0000 7ffbcd847180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T12:44:53.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.235+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f034c04e170 con 0x7f034c0c4410 2023-10-30T12:44:53.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03437fe640 1 -- 172.21.15.8:0/3334738325 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0344004880 con 0x7f034c0c4410 2023-10-30T12:44:53.242 INFO:teuthology.orchestra.run.smithi008.stderr:adjusted tunables profile to default 2023-10-30T12:44:53.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 msgr2=0x7f034c1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f032c008d00 tx=0x7f0344014690 comp rx=0 tx=0).stop 2023-10-30T12:44:53.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 shutdown_connections 2023-10-30T12:44:53.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f034c0c4410 0x7f034c1f5790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f032c006bd0 0x7f034c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 --2- 172.21.15.8:0/3334738325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c003b60 0x7f034c15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 >> 172.21.15.8:0/3334738325 conn(0x7f034c071790 msgr2=0x7f034c158470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:53.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 shutdown_connections 2023-10-30T12:44:53.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:53.239+0000 7f03516f7640 1 -- 172.21.15.8:0/3334738325 wait complete. 2023-10-30T12:44:53.246 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:53.242+0000 7f0d98eb0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T12:44:53.264 INFO:tasks.ceph:check_enable_crimson: False 2023-10-30T12:44:53.264 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-30T12:44:53.264 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:53.265 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-30T12:44:53.286 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:53.286 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-30T12:44:53.349 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:53.349 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-30T12:44:53.413 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:44:53.414 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-30T12:44:53.436 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:53.431+0000 7ffbcd847180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T12:44:53.457 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:53.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-30T12:44:53.471 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:53.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-30T12:44:53.523 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:53.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-30T12:44:53.576 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-30T12:44:53.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-30T12:44:53.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 4a2dea52-4b4c-49b4-87ed-1b82e4a35088 0 2023-10-30T12:44:53.770 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:53.766+0000 7f0d98eb0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T12:44:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.786+0000 7f01a00b1640 1 Processor -- start 2023-10-30T12:44:53.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 -- start start 2023-10-30T12:44:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01980c8150 0x7f0198157040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f01980c8530 0x7f01980c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019806f6b0 con 0x7f01980ca210 2023-10-30T12:44:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019806e6a0 con 0x7f01980c8530 2023-10-30T12:44:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f01a00b1640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0198157580 con 0x7f01980c8150 2023-10-30T12:44:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01980c8150 0x7f0198157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019d64c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f01980ca210 0x7f01980c4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42438/0 (socket says 172.21.15.46:42438) 2023-10-30T12:44:53.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019d64c640 1 -- 172.21.15.46:0/3604951755 learned_addr learned my addr 172.21.15.46:0/3604951755 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-30T12:44:53.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019e64e640 1 --2- 172.21.15.46:0/3604951755 >> v2:172.21.15.8:3300/0 conn(0x7f01980c8530 0x7f01980c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 -- 172.21.15.46:0/3604951755 >> v2:172.21.15.8:3300/0 conn(0x7f01980c8530 msgr2=0x7f01980c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:53.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 --2- 172.21.15.46:0/3604951755 >> v2:172.21.15.8:3300/0 conn(0x7f01980c8530 0x7f01980c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 -- 172.21.15.46:0/3604951755 >> v1:172.21.15.46:6789/0 conn(0x7f01980ca210 legacy=0x7f01980c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 -- 172.21.15.46:0/3604951755 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980ca720 con 0x7f01980c8150 2023-10-30T12:44:53.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019de4d640 1 --2- 172.21.15.46:0/3604951755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8150 0x7f0198157040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0198066480 tx=0x7f019401ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67330c0073e76bbf server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019ce4b640 1 -- 172.21.15.46:0/3604951755 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940a8950 con 0x7f01980c8150 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019ce4b640 1 -- 172.21.15.46:0/3604951755 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01940a8ab0 con 0x7f01980c8150 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.790+0000 7f019ce4b640 1 -- 172.21.15.46:0/3604951755 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940a8e40 con 0x7f01980c8150 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3604951755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8150 msgr2=0x7f0198157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3604951755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8150 0x7f0198157040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0198066480 tx=0x7f019401ec50 comp rx=0 tx=0).stop 2023-10-30T12:44:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3604951755 shutdown_connections 2023-10-30T12:44:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3604951755 >> v2:172.21.15.8:3300/0 conn(0x7f01980c8530 0x7f01980c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3604951755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8150 0x7f0198157040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3604951755 >> 172.21.15.46:0/3604951755 conn(0x7f019806afa0 msgr2=0x7f019806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3604951755 shutdown_connections 2023-10-30T12:44:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3604951755 wait complete. 2023-10-30T12:44:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 Processor -- start 2023-10-30T12:44:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- start start 2023-10-30T12:44:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 0x7f0198153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01980ca210 0x7f01980722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981de2e0 con 0x7f01980ca210 2023-10-30T12:44:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981edab0 con 0x7f01980c8530 2023-10-30T12:44:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c1600 con 0x7f01980c8150 2023-10-30T12:44:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019de4d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 0x7f0198153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60738/0 (socket says 172.21.15.46:60738) 2023-10-30T12:44:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 -- 172.21.15.46:0/3770661269 learned_addr learned my addr 172.21.15.46:0/3770661269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019e64e640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01980ca210 0x7f01980722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 -- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 msgr2=0x7f0198153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 0x7f0198153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 -- 172.21.15.46:0/3770661269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01980ca210 msgr2=0x7f01980722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01980ca210 0x7f01980722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 -- 172.21.15.46:0/3770661269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194009420 con 0x7f01980c8530 2023-10-30T12:44:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019de4d640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 0x7f0198153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f019d64c640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f018800c530 tx=0x7f018800c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188005cf0 con 0x7f01980c8530 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188005e50 con 0x7f01980c8530 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01980c18a0 con 0x7f01980c8530 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188007270 con 0x7f01980c8530 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01980c1e60 con 0x7f01980c8530 2023-10-30T12:44:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f018801c440 con 0x7f01980c8530 2023-10-30T12:44:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.794+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0188012740 con 0x7f01980c8530 2023-10-30T12:44:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.798+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0198067540 con 0x7f01980c8530 2023-10-30T12:44:53.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.806+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0188021060 con 0x7f01980c8530 2023-10-30T12:44:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.874+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4a2dea52-4b4c-49b4-87ed-1b82e4a35088", "id": 0} v 0) v1 -- 0x7f019806eea0 con 0x7f01980c8530 2023-10-30T12:44:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f018e7fc640 1 -- 172.21.15.46:0/3770661269 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a2dea52-4b4c-49b4-87ed-1b82e4a35088", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0188017070 con 0x7f01980c8530 2023-10-30T12:44:53.885 INFO:teuthology.orchestra.run.smithi046.stdout:0 2023-10-30T12:44:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 msgr2=0x7f01981ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f018800c530 tx=0x7f018800c9f0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 shutdown_connections 2023-10-30T12:44:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01980c8530 0x7f01981ed570 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01980ca210 0x7f01980722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 --2- 172.21.15.46:0/3770661269 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01980c8150 0x7f0198153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 >> 172.21.15.46:0/3770661269 conn(0x7f019806afa0 msgr2=0x7f019806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 shutdown_connections 2023-10-30T12:44:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:53.882+0000 7f01a00b1640 1 -- 172.21.15.46:0/3770661269 wait complete. 2023-10-30T12:44:53.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 86288470-58d9-44da-a82b-62eb5d96c002 1 2023-10-30T12:44:53.948 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:53.942+0000 7f0d98eb0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T12:44:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 Processor -- start 2023-10-30T12:44:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 -- start start 2023-10-30T12:44:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21e41597d0 0x7f21e415d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f21e415da70 0x7f21e4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e415cdc0 con 0x7f21e40c5dc0 2023-10-30T12:44:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e406f600 con 0x7f21e415da70 2023-10-30T12:44:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21ea846640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40ca100 con 0x7f21e41597d0 2023-10-30T12:44:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21e41597d0 0x7f21e415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f21e40c5dc0 0x7f21e406eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42440/0 (socket says 172.21.15.46:42440) 2023-10-30T12:44:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e3fff640 1 -- 172.21.15.46:0/2232305852 learned_addr learned my addr 172.21.15.46:0/2232305852 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-30T12:44:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e8de3640 1 --2- 172.21.15.46:0/2232305852 >> v2:172.21.15.8:3300/0 conn(0x7f21e415da70 0x7f21e4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e37fe640 1 -- 172.21.15.46:0/2232305852 >> v2:172.21.15.8:3300/0 conn(0x7f21e415da70 msgr2=0x7f21e4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e37fe640 1 --2- 172.21.15.46:0/2232305852 >> v2:172.21.15.8:3300/0 conn(0x7f21e415da70 0x7f21e4069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e37fe640 1 -- 172.21.15.46:0/2232305852 >> v1:172.21.15.46:6789/0 conn(0x7f21e40c5dc0 legacy=0x7f21e406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e37fe640 1 -- 172.21.15.46:0/2232305852 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4159d60 con 0x7f21e41597d0 2023-10-30T12:44:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.010+0000 7f21e2ffd640 1 -- 172.21.15.46:0/2232305852 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389238418 0 0) 0x7f21e415cdc0 con 0x7f21e40c5dc0 2023-10-30T12:44:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e37fe640 1 --2- 172.21.15.46:0/2232305852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e41597d0 0x7f21e415d530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21e4066e40 tx=0x7f21d801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa17059212b6a31 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e2ffd640 1 -- 172.21.15.46:0/2232305852 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80ab600 con 0x7f21e41597d0 2023-10-30T12:44:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e2ffd640 1 -- 172.21.15.46:0/2232305852 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d80a7050 con 0x7f21e41597d0 2023-10-30T12:44:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e2ffd640 1 -- 172.21.15.46:0/2232305852 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80ab980 con 0x7f21e41597d0 2023-10-30T12:44:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- 172.21.15.46:0/2232305852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e41597d0 msgr2=0x7f21e415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- 172.21.15.46:0/2232305852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e41597d0 0x7f21e415d530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21e4066e40 tx=0x7f21d801c3a0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- 172.21.15.46:0/2232305852 shutdown_connections 2023-10-30T12:44:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- 172.21.15.46:0/2232305852 >> v2:172.21.15.8:3300/0 conn(0x7f21e415da70 0x7f21e4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- 172.21.15.46:0/2232305852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e41597d0 0x7f21e415d530 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- 172.21.15.46:0/2232305852 >> 172.21.15.46:0/2232305852 conn(0x7f21e406afa0 msgr2=0x7f21e406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- 172.21.15.46:0/2232305852 shutdown_connections 2023-10-30T12:44:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- 172.21.15.46:0/2232305852 wait complete. 2023-10-30T12:44:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 Processor -- start 2023-10-30T12:44:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- start start 2023-10-30T12:44:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21e40c5dc0 0x7f21e414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40c53c0 con 0x7f21e40c5dc0 2023-10-30T12:44:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4151090 con 0x7f21e415da70 2023-10-30T12:44:54.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21ea846640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40c4ef0 con 0x7f21e41597d0 2023-10-30T12:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e8de3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e8de3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60752/0 (socket says 172.21.15.46:60752) 2023-10-30T12:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:46934/0 (socket says 172.21.15.46:46934) 2023-10-30T12:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e8de3640 1 -- 172.21.15.46:0/2730530175 learned_addr learned my addr 172.21.15.46:0/2730530175 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.014+0000 7f21e3fff640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21e40c5dc0 0x7f21e414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 -- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 msgr2=0x7f21e41502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 -- 172.21.15.46:0/2730530175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21e40c5dc0 msgr2=0x7f21e414fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21e40c5dc0 0x7f21e414fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 -- 172.21.15.46:0/2730530175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009420 con 0x7f21e415da70 2023-10-30T12:44:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e37fe640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e8de3640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21dc007e90 tx=0x7f21dc00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc014420 con 0x7f21e415da70 2023-10-30T12:44:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc014580 con 0x7f21e415da70 2023-10-30T12:44:54.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e414f8f0 con 0x7f21e415da70 2023-10-30T12:44:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc014930 con 0x7f21e415da70 2023-10-30T12:44:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e414f2e0 con 0x7f21e415da70 2023-10-30T12:44:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f21dc01e460 con 0x7f21e415da70 2023-10-30T12:44:54.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f21dc01d400 con 0x7f21e415da70 2023-10-30T12:44:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e414fa80 con 0x7f21e415da70 2023-10-30T12:44:54.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.018+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f21dc01d660 con 0x7f21e415da70 2023-10-30T12:44:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.098+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "86288470-58d9-44da-a82b-62eb5d96c002", "id": 1} v 0) v1 -- 0x7f21e4067540 con 0x7f21e415da70 2023-10-30T12:44:54.107 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.103+0000 7ffbcd847180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21e0ff9640 1 -- 172.21.15.46:0/2730530175 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "86288470-58d9-44da-a82b-62eb5d96c002", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f21dc01e8a0 con 0x7f21e415da70 2023-10-30T12:44:54.110 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-30T12:44:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 msgr2=0x7f21e4152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21dc007e90 tx=0x7f21dc00e4b0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 shutdown_connections 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21e415da70 0x7f21e4152750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f21e41597d0 0x7f21e41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 --2- 172.21.15.46:0/2730530175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f21e40c5dc0 0x7f21e414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 >> 172.21.15.46:0/2730530175 conn(0x7f21e406afa0 msgr2=0x7f21e4157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 shutdown_connections 2023-10-30T12:44:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.106+0000 7f21ea846640 1 -- 172.21.15.46:0/2730530175 wait complete. 2023-10-30T12:44:54.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 89d39136-f1b7-4184-8d24-66bb377177ed 2 2023-10-30T12:44:54.172 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.167+0000 7ffbcd847180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T12:44:54.235 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.231+0000 7ffbcd847180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T12:44:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.234+0000 7f01c11b6640 1 Processor -- start 2023-10-30T12:44:54.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.234+0000 7f01c11b6640 1 -- start start 2023-10-30T12:44:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f01bc06a4d0 0x7f01bc0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc159fc0 con 0x7f01bc06a0f0 2023-10-30T12:44:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc15d320 con 0x7f01bc06a4d0 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc06f670 con 0x7f01bc0c5f50 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:46946/0 (socket says 172.21.15.46:46946) 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 -- 172.21.15.46:0/2267632794 learned_addr learned my addr 172.21.15.46:0/2267632794 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01ba59c640 1 --2- 172.21.15.46:0/2267632794 >> v2:172.21.15.8:3300/0 conn(0x7f01bc06a4d0 0x7f01bc0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 -- 172.21.15.46:0/2267632794 >> v2:172.21.15.8:3300/0 conn(0x7f01bc06a4d0 msgr2=0x7f01bc0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 --2- 172.21.15.46:0/2267632794 >> v2:172.21.15.8:3300/0 conn(0x7f01bc06a4d0 0x7f01bc0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 -- 172.21.15.46:0/2267632794 >> v1:172.21.15.8:6789/0 conn(0x7f01bc06a0f0 legacy=0x7f01bc0c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 -- 172.21.15.46:0/2267632794 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc06f0b0 con 0x7f01bc0c5f50 2023-10-30T12:44:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01bb59e640 1 --2- 172.21.15.46:0/2267632794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e50 tx=0x7f01b401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c4f973ce977ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01b9d9b640 1 -- 172.21.15.46:0/2267632794 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40a57d0 con 0x7f01bc0c5f50 2023-10-30T12:44:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01b9d9b640 1 -- 172.21.15.46:0/2267632794 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b40a5930 con 0x7f01bc0c5f50 2023-10-30T12:44:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01b9d9b640 1 -- 172.21.15.46:0/2267632794 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40a5cc0 con 0x7f01bc0c5f50 2023-10-30T12:44:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- 172.21.15.46:0/2267632794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 msgr2=0x7f01bc15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 --2- 172.21.15.46:0/2267632794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e50 tx=0x7f01b401ca10 comp rx=0 tx=0).stop 2023-10-30T12:44:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- 172.21.15.46:0/2267632794 shutdown_connections 2023-10-30T12:44:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 --2- 172.21.15.46:0/2267632794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc0c5f50 0x7f01bc15cde0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 --2- 172.21.15.46:0/2267632794 >> v2:172.21.15.8:3300/0 conn(0x7f01bc06a4d0 0x7f01bc0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- 172.21.15.46:0/2267632794 >> 172.21.15.46:0/2267632794 conn(0x7f01bc06afa0 msgr2=0x7f01bc06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- 172.21.15.46:0/2267632794 shutdown_connections 2023-10-30T12:44:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.238+0000 7f01c11b6640 1 -- 172.21.15.46:0/2267632794 wait complete. 2023-10-30T12:44:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 Processor -- start 2023-10-30T12:44:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- start start 2023-10-30T12:44:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 0x7f01bc1efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 0x7f01bc1572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1f6010 con 0x7f01bc0c5f50 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1f4870 con 0x7f01bc06a4d0 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc14e1f0 con 0x7f01bc06a0f0 2023-10-30T12:44:54.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01bb59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 0x7f01bc1572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01bad9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 0x7f01bc1efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60766/0 (socket says 172.21.15.46:60766) 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 -- 172.21.15.46:0/1711429032 learned_addr learned my addr 172.21.15.46:0/1711429032 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 -- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 msgr2=0x7f01bc1efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 0x7f01bc1efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 -- 172.21.15.46:0/1711429032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 msgr2=0x7f01bc1572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 0x7f01bc1572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 -- 172.21.15.46:0/1711429032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4009b30 con 0x7f01bc06a4d0 2023-10-30T12:44:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01bad9d640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 0x7f01bc1efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01bb59e640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 0x7f01bc1572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01ba59c640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00d830 tx=0x7f01ac00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00b6c0 con 0x7f01bc06a4d0 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00b820 con 0x7f01bc06a4d0 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc1f00f0 con 0x7f01bc06a4d0 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00bb90 con 0x7f01bc06a4d0 2023-10-30T12:44:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc1f3f40 con 0x7f01bc06a4d0 2023-10-30T12:44:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f01ac01e760 con 0x7f01bc06a4d0 2023-10-30T12:44:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f01ac014c10 con 0x7f01bc06a4d0 2023-10-30T12:44:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.242+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc159c00 con 0x7f01bc06a4d0 2023-10-30T12:44:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.246+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f01ac0147f0 con 0x7f01bc06a4d0 2023-10-30T12:44:54.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.314+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89d39136-f1b7-4184-8d24-66bb377177ed", "id": 2} v 0) v1 -- 0x7f01bc067540 con 0x7f01bc06a4d0 2023-10-30T12:44:54.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01a37fe640 1 -- 172.21.15.46:0/1711429032 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89d39136-f1b7-4184-8d24-66bb377177ed", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f01ac014a10 con 0x7f01bc06a4d0 2023-10-30T12:44:54.325 INFO:teuthology.orchestra.run.smithi046.stdout:2 2023-10-30T12:44:54.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 msgr2=0x7f01bc1543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00d830 tx=0x7f01ac00dcf0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 shutdown_connections 2023-10-30T12:44:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01bc06a4d0 0x7f01bc1543c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f01bc0c5f50 0x7f01bc1572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 --2- 172.21.15.46:0/1711429032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f01bc06a0f0 0x7f01bc1efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 >> 172.21.15.46:0/1711429032 conn(0x7f01bc06afa0 msgr2=0x7f01bc0caa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 shutdown_connections 2023-10-30T12:44:54.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.322+0000 7f01c11b6640 1 -- 172.21.15.46:0/1711429032 wait complete. 2023-10-30T12:44:54.329 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.327+0000 7ffbcd847180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T12:44:54.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 1d51b4c3-4a06-4f22-9a48-440692bba9bd 3 2023-10-30T12:44:54.393 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.387+0000 7ffbcd847180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T12:44:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.454+0000 7ff54ed05640 1 Processor -- start 2023-10-30T12:44:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.454+0000 7ff54ed05640 1 -- start start 2023-10-30T12:44:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54ed05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff548153ba0 0x7ff5480cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54ed05640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff5480c4810 0x7ff54815a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54ed05640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff548153f80 con 0x7ff5480c4410 2023-10-30T12:44:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54ed05640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff548159df0 con 0x7ff5480c4810 2023-10-30T12:44:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54ed05640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54815a7c0 con 0x7ff548153ba0 2023-10-30T12:44:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff548153ba0 0x7ff5480cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff548153ba0 0x7ff5480cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60770/0 (socket says 172.21.15.46:60770) 2023-10-30T12:44:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 -- 172.21.15.46:0/1407897868 learned_addr learned my addr 172.21.15.46:0/1407897868 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54d2a2640 1 --2- 172.21.15.46:0/1407897868 >> v2:172.21.15.8:3300/0 conn(0x7ff5480c4810 0x7ff54815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 -- 172.21.15.46:0/1407897868 >> v2:172.21.15.8:3300/0 conn(0x7ff5480c4810 msgr2=0x7ff54815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 --2- 172.21.15.46:0/1407897868 >> v2:172.21.15.8:3300/0 conn(0x7ff5480c4810 0x7ff54815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 -- 172.21.15.46:0/1407897868 >> v1:172.21.15.46:6789/0 conn(0x7ff5480c4410 legacy=0x7ff54815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 -- 172.21.15.46:0/1407897868 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5481597c0 con 0x7ff548153ba0 2023-10-30T12:44:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff54caa1640 1 --2- 172.21.15.46:0/1407897868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff548153ba0 0x7ff5480cb4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5480d0170 tx=0x7ff54401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf225bfe5504a450 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff53f7fe640 1 -- 172.21.15.46:0/1407897868 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440ab640 con 0x7ff548153ba0 2023-10-30T12:44:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff53f7fe640 1 -- 172.21.15.46:0/1407897868 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5440a7070 con 0x7ff548153ba0 2023-10-30T12:44:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.458+0000 7ff53f7fe640 1 -- 172.21.15.46:0/1407897868 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440ab9c0 con 0x7ff548153ba0 2023-10-30T12:44:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/1407897868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff548153ba0 msgr2=0x7ff5480cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- 172.21.15.46:0/1407897868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff548153ba0 0x7ff5480cb4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5480d0170 tx=0x7ff54401c440 comp rx=0 tx=0).stop 2023-10-30T12:44:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/1407897868 shutdown_connections 2023-10-30T12:44:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- 172.21.15.46:0/1407897868 >> v2:172.21.15.8:3300/0 conn(0x7ff5480c4810 0x7ff54815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- 172.21.15.46:0/1407897868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff548153ba0 0x7ff5480cb4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/1407897868 >> 172.21.15.46:0/1407897868 conn(0x7ff548071790 msgr2=0x7ff548071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/1407897868 shutdown_connections 2023-10-30T12:44:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/1407897868 wait complete. 2023-10-30T12:44:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 Processor -- start 2023-10-30T12:44:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- start start 2023-10-30T12:44:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5480c4410 0x7ff5480b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5480c4810 0x7ff548142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480ce630 con 0x7ff5480c4410 2023-10-30T12:44:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480cd100 con 0x7ff5480c4810 2023-10-30T12:44:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480cd3a0 con 0x7ff548153ba0 2023-10-30T12:44:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5480c4810 0x7ff548142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:46964/0 (socket says 172.21.15.46:46964) 2023-10-30T12:44:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 -- 172.21.15.46:0/837118666 learned_addr learned my addr 172.21.15.46:0/837118666 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54caa1640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5480c4410 0x7ff5480b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 -- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5480c4810 msgr2=0x7ff548142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5480c4810 0x7ff548142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 -- 172.21.15.46:0/837118666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5480c4410 msgr2=0x7ff5480b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5480c4410 0x7ff5480b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 -- 172.21.15.46:0/837118666 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440094d0 con 0x7ff548153ba0 2023-10-30T12:44:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54d2a2640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff540007e40 tx=0x7ff54000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540014620 con 0x7ff548153ba0 2023-10-30T12:44:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54000ee90 con 0x7ff548153ba0 2023-10-30T12:44:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5480cd680 con 0x7ff548153ba0 2023-10-30T12:44:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400149e0 con 0x7ff548153ba0 2023-10-30T12:44:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5481f56e0 con 0x7ff548153ba0 2023-10-30T12:44:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff54001e460 con 0x7ff548153ba0 2023-10-30T12:44:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.462+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ff54001d850 con 0x7ff548153ba0 2023-10-30T12:44:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.466+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54804e170 con 0x7ff548153ba0 2023-10-30T12:44:54.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.466+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff54001d3c0 con 0x7ff548153ba0 2023-10-30T12:44:54.486 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.483+0000 7ffbcd847180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T12:44:54.549 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.543+0000 7ffbcd847180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T12:44:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.546+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1d51b4c3-4a06-4f22-9a48-440692bba9bd", "id": 3} v 0) v1 -- 0x7ff5480b1f80 con 0x7ff548153ba0 2023-10-30T12:44:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff53d7fa640 1 -- 172.21.15.46:0/837118666 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d51b4c3-4a06-4f22-9a48-440692bba9bd", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff54001db30 con 0x7ff548153ba0 2023-10-30T12:44:54.557 INFO:teuthology.orchestra.run.smithi046.stdout:3 2023-10-30T12:44:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 msgr2=0x7ff5480b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff540007e40 tx=0x7ff54000e4b0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 shutdown_connections 2023-10-30T12:44:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5480c4810 0x7ff548142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff548153ba0 0x7ff5480b3240 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 --2- 172.21.15.46:0/837118666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5480c4410 0x7ff5480b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 >> 172.21.15.46:0/837118666 conn(0x7ff548071790 msgr2=0x7ff548146e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 shutdown_connections 2023-10-30T12:44:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.554+0000 7ff54ed05640 1 -- 172.21.15.46:0/837118666 wait complete. 2023-10-30T12:44:54.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new abceded1-2ec0-4352-bb15-4eb891564cdf 4 2023-10-30T12:44:54.582 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.578+0000 7f0d98eb0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T12:44:54.640 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.634+0000 7f0d98eb0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T12:44:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 Processor -- start 2023-10-30T12:44:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 -- start start 2023-10-30T12:44:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f57ec0c3580 0x7f57ec157df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57ec158330 0x7f57ec162040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec0c6460 con 0x7f57ec0c3960 2023-10-30T12:44:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec1576c0 con 0x7f57ec0c3580 2023-10-30T12:44:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f4345640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec157960 con 0x7f57ec158330 2023-10-30T12:44:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f28e2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57ec158330 0x7f57ec162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f28e2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57ec158330 0x7f57ec162040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60794/0 (socket says 172.21.15.46:60794) 2023-10-30T12:44:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f28e2640 1 -- 172.21.15.46:0/4000346239 learned_addr learned my addr 172.21.15.46:0/4000346239 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.682+0000 7f57f20e1640 1 --2- 172.21.15.46:0/4000346239 >> v2:172.21.15.8:3300/0 conn(0x7f57ec0c3580 0x7f57ec157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f28e2640 1 -- 172.21.15.46:0/4000346239 >> v2:172.21.15.8:3300/0 conn(0x7f57ec0c3580 msgr2=0x7f57ec157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f28e2640 1 --2- 172.21.15.46:0/4000346239 >> v2:172.21.15.8:3300/0 conn(0x7f57ec0c3580 0x7f57ec157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f28e2640 1 -- 172.21.15.46:0/4000346239 >> v1:172.21.15.8:6789/0 conn(0x7f57ec0c3960 legacy=0x7f57ec1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f28e2640 1 -- 172.21.15.46:0/4000346239 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0c5ee0 con 0x7f57ec158330 2023-10-30T12:44:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f28e2640 1 --2- 172.21.15.46:0/4000346239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec158330 0x7f57ec162040 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0c8fa0 tx=0x7f57e401c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b64f68ad55a2bff server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f10df640 1 -- 172.21.15.46:0/4000346239 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40a5670 con 0x7f57ec158330 2023-10-30T12:44:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f10df640 1 -- 172.21.15.46:0/4000346239 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e40a57d0 con 0x7f57ec158330 2023-10-30T12:44:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.686+0000 7f57f10df640 1 -- 172.21.15.46:0/4000346239 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40a5b60 con 0x7f57ec158330 2023-10-30T12:44:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/4000346239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec158330 msgr2=0x7f57ec162040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- 172.21.15.46:0/4000346239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec158330 0x7f57ec162040 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0c8fa0 tx=0x7f57e401c9d0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/4000346239 shutdown_connections 2023-10-30T12:44:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- 172.21.15.46:0/4000346239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec158330 0x7f57ec162040 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- 172.21.15.46:0/4000346239 >> v2:172.21.15.8:3300/0 conn(0x7f57ec0c3580 0x7f57ec157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/4000346239 >> 172.21.15.46:0/4000346239 conn(0x7f57ec06e1a0 msgr2=0x7f57ec06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/4000346239 shutdown_connections 2023-10-30T12:44:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/4000346239 wait complete. 2023-10-30T12:44:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 Processor -- start 2023-10-30T12:44:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- start start 2023-10-30T12:44:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 0x7f57ec150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec158330 0x7f57ec14e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1f0bf0 con 0x7f57ec158330 2023-10-30T12:44:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1f0690 con 0x7f57ec0c3580 2023-10-30T12:44:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec14cdb0 con 0x7f57ec0c3960 2023-10-30T12:44:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:46978/0 (socket says 172.21.15.46:46978) 2023-10-30T12:44:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 -- 172.21.15.46:0/3841554274 learned_addr learned my addr 172.21.15.46:0/3841554274 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f20e1640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 0x7f57ec150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f28e2640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec158330 0x7f57ec14e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 -- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 msgr2=0x7f57ec150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 0x7f57ec150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 -- 172.21.15.46:0/3841554274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec158330 msgr2=0x7f57ec14e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec158330 0x7f57ec14e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 -- 172.21.15.46:0/3841554274 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4009b30 con 0x7f57ec0c3960 2023-10-30T12:44:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f20e1640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 0x7f57ec150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f18e0640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c530 tx=0x7f57dc00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc005220 con 0x7f57ec0c3960 2023-10-30T12:44:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc005380 con 0x7f57ec0c3960 2023-10-30T12:44:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec1f0930 con 0x7f57ec0c3960 2023-10-30T12:44:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.690+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc007270 con 0x7f57ec0c3960 2023-10-30T12:44:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec14d330 con 0x7f57ec0c3960 2023-10-30T12:44:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f57dc0067e0 con 0x7f57ec0c3960 2023-10-30T12:44:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f57dc012950 con 0x7f57ec0c3960 2023-10-30T12:44:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec067540 con 0x7f57ec0c3960 2023-10-30T12:44:54.704 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f0d98eb0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T12:44:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.694+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f57dc016020 con 0x7f57ec0c3960 2023-10-30T12:44:54.714 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.711+0000 7ffbcd847180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T12:44:54.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.762+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd new", "uuid": "abceded1-2ec0-4352-bb15-4eb891564cdf", "id": 4} v 0) v1 -- 0x7f57ec06e790 con 0x7f57ec0c3960 2023-10-30T12:44:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57e2ffd640 1 -- 172.21.15.46:0/3841554274 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "abceded1-2ec0-4352-bb15-4eb891564cdf", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57dc02a0b0 con 0x7f57ec0c3960 2023-10-30T12:44:54.774 INFO:teuthology.orchestra.run.smithi046.stdout:4 2023-10-30T12:44:54.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 msgr2=0x7f57ec14db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c530 tx=0x7f57dc00c9f0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 shutdown_connections 2023-10-30T12:44:54.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57ec158330 0x7f57ec14e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f57ec0c3960 0x7f57ec14db30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 --2- 172.21.15.46:0/3841554274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57ec0c3580 0x7f57ec150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 >> 172.21.15.46:0/3841554274 conn(0x7f57ec06e1a0 msgr2=0x7f57ec0c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 shutdown_connections 2023-10-30T12:44:54.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.770+0000 7f57f4345640 1 -- 172.21.15.46:0/3841554274 wait complete. 2023-10-30T12:44:54.779 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.774+0000 7f0d98eb0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T12:44:54.787 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:54.783+0000 7ffbcd847180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T12:44:54.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 7b7814ee-3a48-48e1-8add-ad5f8090df2c 5 2023-10-30T12:44:54.834 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.830+0000 7f0d98eb0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T12:44:54.917 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.914+0000 7f0d98eb0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T12:44:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.946+0000 7f3bd7778640 1 Processor -- start 2023-10-30T12:44:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.946+0000 7f3bd7778640 1 -- start start 2023-10-30T12:44:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd7778640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bd01508f0 0x7f3bd00c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd7778640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 0x7f3bd015a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd7778640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0071710 con 0x7f3bd0070680 2023-10-30T12:44:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd7778640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00c4df0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd7778640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd015a560 con 0x7f3bd01508f0 2023-10-30T12:44:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bd01508f0 0x7f3bd00c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bd01508f0 0x7f3bd00c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60814/0 (socket says 172.21.15.46:60814) 2023-10-30T12:44:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 -- 172.21.15.46:0/1633988067 learned_addr learned my addr 172.21.15.46:0/1633988067 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/1633988067 >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 0x7f3bd015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 -- 172.21.15.46:0/1633988067 >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 msgr2=0x7f3bd015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 --2- 172.21.15.46:0/1633988067 >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 0x7f3bd015a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 -- 172.21.15.46:0/1633988067 >> v1:172.21.15.8:6789/0 conn(0x7f3bd0070680 legacy=0x7f3bd006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5514640 1 -- 172.21.15.46:0/1633988067 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00c4830 con 0x7f3bd01508f0 2023-10-30T12:44:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.950+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/1633988067 >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 0x7f3bd015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:44:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd5514640 1 --2- 172.21.15.46:0/1633988067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd01508f0 0x7f3bd00c4240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ad60 tx=0x7f3bc801ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e93a57a1c5b8ba8d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bcffff640 1 -- 172.21.15.46:0/1633988067 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80a89b0 con 0x7f3bd01508f0 2023-10-30T12:44:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bcffff640 1 -- 172.21.15.46:0/1633988067 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc80a8b10 con 0x7f3bd01508f0 2023-10-30T12:44:54.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bcffff640 1 -- 172.21.15.46:0/1633988067 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80a8ea0 con 0x7f3bd01508f0 2023-10-30T12:44:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- 172.21.15.46:0/1633988067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd01508f0 msgr2=0x7f3bd00c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 --2- 172.21.15.46:0/1633988067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd01508f0 0x7f3bd00c4240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ad60 tx=0x7f3bc801ecb0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- 172.21.15.46:0/1633988067 shutdown_connections 2023-10-30T12:44:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 --2- 172.21.15.46:0/1633988067 >> v2:172.21.15.8:3300/0 conn(0x7f3bd0159bb0 0x7f3bd015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 --2- 172.21.15.46:0/1633988067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd01508f0 0x7f3bd00c4240 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- 172.21.15.46:0/1633988067 >> 172.21.15.46:0/1633988067 conn(0x7f3bd0069e90 msgr2=0x7f3bd006a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- 172.21.15.46:0/1633988067 shutdown_connections 2023-10-30T12:44:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- 172.21.15.46:0/1633988067 wait complete. 2023-10-30T12:44:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 Processor -- start 2023-10-30T12:44:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.954+0000 7f3bd7778640 1 -- start start 2023-10-30T12:44:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 0x7f3bd01496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 0x7f3bd01577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd014a140 con 0x7f3bd01508f0 2023-10-30T12:44:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0147c10 con 0x7f3bd0070680 2023-10-30T12:44:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01ec9d0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5514640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5514640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60828/0 (socket says 172.21.15.46:60828) 2023-10-30T12:44:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5514640 1 -- 172.21.15.46:0/3257122805 learned_addr learned my addr 172.21.15.46:0/3257122805 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 0x7f3bd01577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd4d13640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 0x7f3bd01496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 -- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 msgr2=0x7f3bd01ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 -- 172.21.15.46:0/3257122805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 msgr2=0x7f3bd01496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 0x7f3bd01496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 -- 172.21.15.46:0/3257122805 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc80094d0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5514640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:44:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd4d13640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 0x7f3bd01496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd5d15640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 0x7f3bd01577a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000cc90 tx=0x7f3bc000a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000cea0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc000f6a0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd0149190 con 0x7f3bd0159bb0 2023-10-30T12:44:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000fa50 con 0x7f3bd0159bb0 2023-10-30T12:44:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd01472e0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3bc000b040 con 0x7f3bd0159bb0 2023-10-30T12:44:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3bc0018d10 con 0x7f3bd0159bb0 2023-10-30T12:44:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.958+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0147dc0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:54.962+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3bd0147dc0 con 0x7f3bd0159bb0 2023-10-30T12:44:54.976 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:54.970+0000 7f0d98eb0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T12:44:55.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.026+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7b7814ee-3a48-48e1-8add-ad5f8090df2c", "id": 5} v 0) v1 -- 0x7f3bd0067540 con 0x7f3bd0159bb0 2023-10-30T12:44:55.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bcdffb640 1 -- 172.21.15.46:0/3257122805 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b7814ee-3a48-48e1-8add-ad5f8090df2c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3bc00184d0 con 0x7f3bd0159bb0 2023-10-30T12:44:55.038 INFO:teuthology.orchestra.run.smithi046.stdout:5 2023-10-30T12:44:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 msgr2=0x7f3bd01577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 0x7f3bd01577a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000cc90 tx=0x7f3bc000a650 comp rx=0 tx=0).stop 2023-10-30T12:44:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 shutdown_connections 2023-10-30T12:44:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd0159bb0 0x7f3bd01577a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bd01508f0 0x7f3bd01496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 --2- 172.21.15.46:0/3257122805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd0070680 0x7f3bd01ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 >> 172.21.15.46:0/3257122805 conn(0x7f3bd0069e90 msgr2=0x7f3bd006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 shutdown_connections 2023-10-30T12:44:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.034+0000 7f3bd7778640 1 -- 172.21.15.46:0/3257122805 wait complete. 2023-10-30T12:44:55.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 3898a740-c477-4f12-b71b-a60cdf7bd407 6 2023-10-30T12:44:55.124 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:55.122+0000 7f0d98eb0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T12:44:55.188 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:55.186+0000 7f0d98eb0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T12:44:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 Processor -- start 2023-10-30T12:44:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 -- start start 2023-10-30T12:44:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff8159a70 con 0x7f4ff80c65a0 2023-10-30T12:44:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff806e210 con 0x7f4ff80706f0 2023-10-30T12:44:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffee71640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff806e4f0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffcc0d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ff7fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4ff80706f0 0x7f4ff8153770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42462/0 (socket says 172.21.15.46:42462) 2023-10-30T12:44:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ff7fff640 1 -- 172.21.15.46:0/66842475 learned_addr learned my addr 172.21.15.46:0/66842475 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-30T12:44:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffcc0d640 1 -- 172.21.15.46:0/66842475 >> v1:172.21.15.46:6789/0 conn(0x7f4ff80706f0 legacy=0x7f4ff8153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ff77fe640 1 -- 172.21.15.46:0/66842475 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970685801 0 0) 0x7f4ff806e210 con 0x7f4ff80706f0 2023-10-30T12:44:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffcc0d640 1 -- 172.21.15.46:0/66842475 >> v1:172.21.15.8:6789/0 conn(0x7f4ff80c65a0 legacy=0x7f4ff8159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffcc0d640 1 -- 172.21.15.46:0/66842475 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff806e7d0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.206+0000 7f4ffcc0d640 1 --2- 172.21.15.46:0/66842475 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff80c7ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0097f0 tx=0x7f4fec01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26feaaab972cf30 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff77fe640 1 -- 172.21.15.46:0/66842475 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0a89b0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff77fe640 1 -- 172.21.15.46:0/66842475 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec0a8b10 con 0x7f4ff80c61c0 2023-10-30T12:44:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff77fe640 1 -- 172.21.15.46:0/66842475 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0a8ea0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/66842475 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 msgr2=0x7f4ff80c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 --2- 172.21.15.46:0/66842475 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff80c7ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0097f0 tx=0x7f4fec01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/66842475 shutdown_connections 2023-10-30T12:44:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 --2- 172.21.15.46:0/66842475 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff80c7ff0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/66842475 >> 172.21.15.46:0/66842475 conn(0x7f4ff8069e90 msgr2=0x7f4ff806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/66842475 shutdown_connections 2023-10-30T12:44:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/66842475 wait complete. 2023-10-30T12:44:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 Processor -- start 2023-10-30T12:44:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- start start 2023-10-30T12:44:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 0x7f4ff814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff80c65a0 0x7f4ff80ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81ef290 con 0x7f4ff80706f0 2023-10-30T12:44:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff814ec70 con 0x7f4ff80c65a0 2023-10-30T12:44:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8157300 con 0x7f4ff80c61c0 2023-10-30T12:44:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:47002/0 (socket says 172.21.15.46:47002) 2023-10-30T12:44:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 -- 172.21.15.46:0/125070643 learned_addr learned my addr 172.21.15.46:0/125070643 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffd40e640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff80c65a0 0x7f4ff80ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffcc0d640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 0x7f4ff814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 -- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff80c65a0 msgr2=0x7f4ff80ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff80c65a0 0x7f4ff80ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 -- 172.21.15.46:0/125070643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 msgr2=0x7f4ff814c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 0x7f4ff814c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 -- 172.21.15.46:0/125070643 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec009420 con 0x7f4ff80c61c0 2023-10-30T12:44:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffcc0d640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 0x7f4ff814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff7fff640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4a0 tx=0x7f4fe400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40074e0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4007640 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff81ee8f0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4004a30 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81eeeb0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4fe4012400 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.210+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4fe401b950 con 0x7f4ff80c61c0 2023-10-30T12:44:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.214+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff804e170 con 0x7f4ff80c61c0 2023-10-30T12:44:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.218+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4fe4018070 con 0x7f4ff80c61c0 2023-10-30T12:44:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.286+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3898a740-c477-4f12-b71b-a60cdf7bd407", "id": 6} v 0) v1 -- 0x7f4ff80c6a30 con 0x7f4ff80c61c0 2023-10-30T12:44:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ff57fa640 1 -- 172.21.15.46:0/125070643 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3898a740-c477-4f12-b71b-a60cdf7bd407", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4fe401bcc0 con 0x7f4ff80c61c0 2023-10-30T12:44:55.293 INFO:teuthology.orchestra.run.smithi046.stdout:6 2023-10-30T12:44:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 msgr2=0x7f4ff814c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4a0 tx=0x7f4fe400c960 comp rx=0 tx=0).stop 2023-10-30T12:44:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 shutdown_connections 2023-10-30T12:44:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff80c65a0 0x7f4ff80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ff80c61c0 0x7f4ff814c580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 --2- 172.21.15.46:0/125070643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ff80706f0 0x7f4ff814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 >> 172.21.15.46:0/125070643 conn(0x7f4ff8069e90 msgr2=0x7f4ff814dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 shutdown_connections 2023-10-30T12:44:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.290+0000 7f4ffee71640 1 -- 172.21.15.46:0/125070643 wait complete. 2023-10-30T12:44:55.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new d0befb09-63a8-4ab8-8464-efdf1c8fc231 7 2023-10-30T12:44:55.413 INFO:tasks.ceph.mgr.y.smithi008.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-30T12:44:55.414 INFO:tasks.ceph.mgr.y.smithi008.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-30T12:44:55.414 INFO:tasks.ceph.mgr.y.smithi008.stderr: from numpy import show_config as show_numpy_config 2023-10-30T12:44:55.459 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:55.454+0000 7ffbcd847180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T12:44:55.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.466+0000 7f58b1775640 1 Processor -- start 2023-10-30T12:44:55.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.466+0000 7f58b1775640 1 -- start start 2023-10-30T12:44:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 0x7f58ac0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac1581f0 con 0x7f58ac0c74a0 2023-10-30T12:44:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0cb9f0 con 0x7f58ac0c1020 2023-10-30T12:44:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1576d0 con 0x7f58ac151a20 2023-10-30T12:44:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aaffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 0x7f58ac0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aa7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58ac0c1020 0x7f58ac072080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42476/0 (socket says 172.21.15.46:42476) 2023-10-30T12:44:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aa7fc640 1 -- 172.21.15.46:0/1451105793 learned_addr learned my addr 172.21.15.46:0/1451105793 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-30T12:44:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58a9ffb640 1 -- 172.21.15.46:0/1451105793 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677037283 0 0) 0x7f58ac0cb9f0 con 0x7f58ac0c1020 2023-10-30T12:44:55.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aaffd640 1 -- 172.21.15.46:0/1451105793 >> v1:172.21.15.46:6789/0 conn(0x7f58ac0c1020 legacy=0x7f58ac072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aaffd640 1 -- 172.21.15.46:0/1451105793 >> v1:172.21.15.8:6789/0 conn(0x7f58ac0c74a0 legacy=0x7f58ac161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:55.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aaffd640 1 -- 172.21.15.46:0/1451105793 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac151e00 con 0x7f58ac151a20 2023-10-30T12:44:55.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58aaffd640 1 --2- 172.21.15.46:0/1451105793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 0x7f58ac0cb4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f589c0097f0 tx=0x7f589c01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8fa757502aa04a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58a9ffb640 1 -- 172.21.15.46:0/1451105793 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a89b0 con 0x7f58ac151a20 2023-10-30T12:44:55.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58a9ffb640 1 -- 172.21.15.46:0/1451105793 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0a8b10 con 0x7f58ac151a20 2023-10-30T12:44:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58a9ffb640 1 -- 172.21.15.46:0/1451105793 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a8ea0 con 0x7f58ac151a20 2023-10-30T12:44:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- 172.21.15.46:0/1451105793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 msgr2=0x7f58ac0cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 --2- 172.21.15.46:0/1451105793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 0x7f58ac0cb4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f589c0097f0 tx=0x7f589c01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- 172.21.15.46:0/1451105793 shutdown_connections 2023-10-30T12:44:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 --2- 172.21.15.46:0/1451105793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac151a20 0x7f58ac0cb4b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- 172.21.15.46:0/1451105793 >> 172.21.15.46:0/1451105793 conn(0x7f58ac06da40 msgr2=0x7f58ac06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- 172.21.15.46:0/1451105793 shutdown_connections 2023-10-30T12:44:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.470+0000 7f58b1775640 1 -- 172.21.15.46:0/1451105793 wait complete. 2023-10-30T12:44:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 Processor -- start 2023-10-30T12:44:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- start start 2023-10-30T12:44:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 0x7f58ac155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac151a20 0x7f58ac1ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac157110 con 0x7f58ac0c74a0 2023-10-30T12:44:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac148620 con 0x7f58ac151a20 2023-10-30T12:44:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1eceb0 con 0x7f58ac0c1020 2023-10-30T12:44:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.46:47030/0 (socket says 172.21.15.46:47030) 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 -- 172.21.15.46:0/3083732848 learned_addr learned my addr 172.21.15.46:0/3083732848 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58ab7fe640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac151a20 0x7f58ac1ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aa7fc640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 0x7f58ac155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 -- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac151a20 msgr2=0x7f58ac1ed3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac151a20 0x7f58ac1ed3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 -- 172.21.15.46:0/3083732848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 msgr2=0x7f58ac155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 0x7f58ac155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 -- 172.21.15.46:0/3083732848 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c009420 con 0x7f58ac0c1020 2023-10-30T12:44:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aa7fc640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 0x7f58ac155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:44:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58aaffd640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f589c01b8c0 tx=0x7f589c01ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b9630 con 0x7f58ac0c1020 2023-10-30T12:44:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0b97b0 con 0x7f58ac0c1020 2023-10-30T12:44:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b9b60 con 0x7f58ac0c1020 2023-10-30T12:44:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac1488e0 con 0x7f58ac0c1020 2023-10-30T12:44:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac149200 con 0x7f58ac0c1020 2023-10-30T12:44:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f589c0b7040 con 0x7f58ac0c1020 2023-10-30T12:44:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f589c0c4c80 con 0x7f58ac0c1020 2023-10-30T12:44:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.474+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac149390 con 0x7f58ac0c1020 2023-10-30T12:44:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.478+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f589c0b5070 con 0x7f58ac0c1020 2023-10-30T12:44:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.586+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d0befb09-63a8-4ab8-8464-efdf1c8fc231", "id": 7} v 0) v1 -- 0x7f58ac04e170 con 0x7f58ac0c1020 2023-10-30T12:44:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.590+0000 7f58937fe640 1 -- 172.21.15.46:0/3083732848 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0befb09-63a8-4ab8-8464-efdf1c8fc231", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f589c0c4460 con 0x7f58ac0c1020 2023-10-30T12:44:55.600 INFO:teuthology.orchestra.run.smithi046.stdout:7 2023-10-30T12:44:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 msgr2=0x7f58ac1f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f589c01b8c0 tx=0x7f589c01ba50 comp rx=0 tx=0).stop 2023-10-30T12:44:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 shutdown_connections 2023-10-30T12:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac151a20 0x7f58ac1ed3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58ac0c74a0 0x7f58ac155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 --2- 172.21.15.46:0/3083732848 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58ac0c1020 0x7f58ac1f0e30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 >> 172.21.15.46:0/3083732848 conn(0x7f58ac06da40 msgr2=0x7f58ac0c5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 shutdown_connections 2023-10-30T12:44:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-30T12:44:55.594+0000 7f58b1775640 1 -- 172.21.15.46:0/3083732848 wait complete. 2023-10-30T12:44:55.603 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:55.598+0000 7ffbcd847180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:44:55.615 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-30T12:44:55.615 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-30T12:44:55.619 INFO:tasks.ceph.osd.0:Started 2023-10-30T12:44:55.619 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-30T12:44:55.619 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-30T12:44:55.623 INFO:tasks.ceph.osd.1:Started 2023-10-30T12:44:55.623 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-30T12:44:55.623 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-30T12:44:55.627 INFO:tasks.ceph.osd.2:Started 2023-10-30T12:44:55.628 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-30T12:44:55.628 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-30T12:44:55.633 INFO:tasks.ceph.osd.3:Started 2023-10-30T12:44:55.633 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-30T12:44:55.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-30T12:44:55.636 INFO:tasks.ceph.osd.4:Started 2023-10-30T12:44:55.636 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-30T12:44:55.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-30T12:44:55.639 INFO:tasks.ceph.osd.5:Started 2023-10-30T12:44:55.641 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-30T12:44:55.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-30T12:44:55.643 INFO:tasks.ceph.osd.6:Started 2023-10-30T12:44:55.643 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-30T12:44:55.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-30T12:44:55.645 INFO:tasks.ceph.osd.7:Started 2023-10-30T12:44:55.646 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:44:55.787 INFO:tasks.ceph.mgr.x.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-30T12:44:55.787 INFO:tasks.ceph.mgr.x.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-30T12:44:55.788 INFO:tasks.ceph.mgr.x.smithi046.stderr: from numpy import show_config as show_numpy_config 2023-10-30T12:44:55.832 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:55.826+0000 7ffbcd847180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T12:44:55.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 Processor -- start 2023-10-30T12:44:55.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 -- start start 2023-10-30T12:44:55.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd1e00c4a10 0x7fd1e015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e0159f20 con 0x7fd1e0153c80 2023-10-30T12:44:55.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e015d620 con 0x7fd1e00c42a0 2023-10-30T12:44:55.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1e6565640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e015dec0 con 0x7fd1e00c4a10 2023-10-30T12:44:55.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1dffff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd1e0153c80 0x7fd1e015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:58604/0 (socket says 172.21.15.8:58604) 2023-10-30T12:44:55.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1dffff640 1 -- 172.21.15.8:0/774261670 learned_addr learned my addr 172.21.15.8:0/774261670 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:44:55.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1df7fe640 1 --2- 172.21.15.8:0/774261670 >> v2:172.21.15.8:3300/0 conn(0x7fd1e00c4a10 0x7fd1e015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1df7fe640 1 -- 172.21.15.8:0/774261670 >> v1:172.21.15.46:6789/0 conn(0x7fd1e00c42a0 legacy=0x7fd1e00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1df7fe640 1 -- 172.21.15.8:0/774261670 >> v1:172.21.15.8:6789/0 conn(0x7fd1e0153c80 legacy=0x7fd1e015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.842+0000 7fd1df7fe640 1 -- 172.21.15.8:0/774261670 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e015d900 con 0x7fd1e00c4a10 2023-10-30T12:44:55.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1df7fe640 1 --2- 172.21.15.8:0/774261670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e015cf90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40097c0 tx=0x7fd1d401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca40f503b42afd76 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1deffd640 1 -- 172.21.15.8:0/774261670 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40a8970 con 0x7fd1e00c4a10 2023-10-30T12:44:55.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1deffd640 1 -- 172.21.15.8:0/774261670 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d40a8ad0 con 0x7fd1e00c4a10 2023-10-30T12:44:55.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1deffd640 1 -- 172.21.15.8:0/774261670 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40a8e60 con 0x7fd1e00c4a10 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 -- 172.21.15.8:0/774261670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 msgr2=0x7fd1e015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 --2- 172.21.15.8:0/774261670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e015cf90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40097c0 tx=0x7fd1d401ec70 comp rx=0 tx=0).stop 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 -- 172.21.15.8:0/774261670 shutdown_connections 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 --2- 172.21.15.8:0/774261670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e015cf90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 -- 172.21.15.8:0/774261670 >> 172.21.15.8:0/774261670 conn(0x7fd1e00715c0 msgr2=0x7fd1e00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 -- 172.21.15.8:0/774261670 shutdown_connections 2023-10-30T12:44:55.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.846+0000 7fd1e6565640 1 -- 172.21.15.8:0/774261670 wait complete. 2023-10-30T12:44:55.854 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:55.846+0000 7f0d98eb0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T12:44:55.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.850+0000 7fd1e6565640 1 Processor -- start 2023-10-30T12:44:55.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.850+0000 7fd1e6565640 1 -- start start 2023-10-30T12:44:55.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.850+0000 7fd1e6565640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd1e00c42a0 0x7fd1e0149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.850+0000 7fd1e6565640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1e0153c80 0x7fd1e00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:55.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e015cc80 con 0x7fd1e00c4a10 2023-10-30T12:44:55.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00b3470 con 0x7fd1e0153c80 2023-10-30T12:44:55.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00b3b60 con 0x7fd1e00c42a0 2023-10-30T12:44:55.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55710/0 (socket says 172.21.15.8:55710) 2023-10-30T12:44:55.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 -- 172.21.15.8:0/3867071099 learned_addr learned my addr 172.21.15.8:0/3867071099 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:44:55.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dffff640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd1e00c42a0 0x7fd1e0149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e4b02640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1e0153c80 0x7fd1e00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:55.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 -- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd1e00c42a0 msgr2=0x7fd1e0149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd1e00c42a0 0x7fd1e0149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 -- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1e0153c80 msgr2=0x7fd1e00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1e0153c80 0x7fd1e00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 -- 172.21.15.8:0/3867071099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4009420 con 0x7fd1e00c4a10 2023-10-30T12:44:55.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1df7fe640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40b3040 tx=0x7fd1d401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:55.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40a8970 con 0x7fd1e00c4a10 2023-10-30T12:44:55.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d40b9410 con 0x7fd1e00c4a10 2023-10-30T12:44:55.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40b97e0 con 0x7fd1e00c4a10 2023-10-30T12:44:55.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e015d170 con 0x7fd1e00c4a10 2023-10-30T12:44:55.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e015ce90 con 0x7fd1e00c4a10 2023-10-30T12:44:55.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd1d40b7040 con 0x7fd1e00c4a10 2023-10-30T12:44:55.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd1d40c6460 con 0x7fd1e00c4a10 2023-10-30T12:44:55.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.854+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e004e170 con 0x7fd1e00c4a10 2023-10-30T12:44:55.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.858+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd1d40b5070 con 0x7fd1e00c4a10 2023-10-30T12:44:55.914 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:55.910+0000 7ffbcd847180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T12:44:55.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1e00cad40 con 0x7fd1e00c4a10 2023-10-30T12:44:55.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1dcff9640 1 -- 172.21.15.8:0/3867071099 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd1d40b9c40 con 0x7fd1e00c4a10 2023-10-30T12:44:55.923 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:44:55.924 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":10,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:44:55.596742+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:44:55.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 msgr2=0x7fd1e014a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:55.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40b3040 tx=0x7fd1d401ba10 comp rx=0 tx=0).stop 2023-10-30T12:44:55.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 shutdown_connections 2023-10-30T12:44:55.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1e0153c80 0x7fd1e00b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e00c4a10 0x7fd1e014a070 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 --2- 172.21.15.8:0/3867071099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd1e00c42a0 0x7fd1e0149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:55.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 >> 172.21.15.8:0/3867071099 conn(0x7fd1e00715c0 msgr2=0x7fd1e00c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:55.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 shutdown_connections 2023-10-30T12:44:55.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:44:55.918+0000 7fd1e6565640 1 -- 172.21.15.8:0/3867071099 wait complete. 2023-10-30T12:44:55.943 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-30T12:44:55.944 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-30T12:44:56.017 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:56.010+0000 7f0d98eb0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:44:56.061 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:56.058+0000 7ffbcd847180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:44:56.179 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:56.170+0000 7f0d98eb0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T12:44:56.242 INFO:tasks.ceph.osd.5.smithi046.stderr:2023-10-30T12:44:56.234+0000 7f68c398c640 -1 Falling back to public interface 2023-10-30T12:44:56.246 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:56.238+0000 7f0d98eb0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T12:44:56.257 INFO:tasks.ceph.osd.7.smithi046.stderr:2023-10-30T12:44:56.250+0000 7fb29427c640 -1 Falling back to public interface 2023-10-30T12:44:56.257 INFO:tasks.ceph.osd.4.smithi046.stderr:2023-10-30T12:44:56.250+0000 7f093fafa640 -1 Falling back to public interface 2023-10-30T12:44:56.269 INFO:tasks.ceph.osd.6.smithi046.stderr:2023-10-30T12:44:56.262+0000 7fc67c20d640 -1 Falling back to public interface 2023-10-30T12:44:56.299 INFO:tasks.ceph.osd.3.smithi008.stderr:2023-10-30T12:44:56.294+0000 7f3e2a6a9640 -1 Falling back to public interface 2023-10-30T12:44:56.299 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-30T12:44:56.294+0000 7ff6c25fd640 -1 Falling back to public interface 2023-10-30T12:44:56.300 INFO:tasks.ceph.osd.2.smithi008.stderr:2023-10-30T12:44:56.294+0000 7f0fe5fb6640 -1 Falling back to public interface 2023-10-30T12:44:56.300 INFO:tasks.ceph.osd.1.smithi008.stderr:2023-10-30T12:44:56.294+0000 7f3ad7451640 -1 Falling back to public interface 2023-10-30T12:44:56.378 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:56.370+0000 7f0d98eb0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T12:44:56.750 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:44:56.783 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:56.778+0000 7ffbcd847180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T12:44:56.880 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:56.870+0000 7ffbcd847180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T12:44:56.883 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 Processor -- start 2023-10-30T12:44:56.896 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 -- start start 2023-10-30T12:44:56.896 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe108153b30 0x7fe1080c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.897 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe1080cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.897 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1080caa30 con 0x7fe1080ca040 2023-10-30T12:44:56.897 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe108153f10 con 0x7fe108153b30 2023-10-30T12:44:56.897 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10f73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10815cde0 con 0x7fe10815a1f0 2023-10-30T12:44:56.897 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10ccd9640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe1080ca040 0x7fe10815d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:58606/0 (socket says 172.21.15.8:58606) 2023-10-30T12:44:56.898 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10ccd9640 1 -- 172.21.15.8:0/329690594 learned_addr learned my addr 172.21.15.8:0/329690594 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:44:56.898 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10dcdb640 1 --2- 172.21.15.8:0/329690594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe1080cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.899 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.878+0000 7fe10d4da640 1 --2- 172.21.15.8:0/329690594 >> v2:172.21.15.46:3300/0 conn(0x7fe108153b30 0x7fe1080c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.899 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe0fffff640 1 -- 172.21.15.8:0/329690594 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356949956 0 0) 0x7fe0f8001120 con 0x7fe1080ca040 2023-10-30T12:44:56.901 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe0fffff640 1 -- 172.21.15.8:0/329690594 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f4003530 con 0x7fe1080ca040 2023-10-30T12:44:56.902 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe10dcdb640 1 -- 172.21.15.8:0/329690594 >> v2:172.21.15.46:3300/0 conn(0x7fe108153b30 msgr2=0x7fe1080c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:44:56.902 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe10dcdb640 1 --2- 172.21.15.8:0/329690594 >> v2:172.21.15.46:3300/0 conn(0x7fe108153b30 0x7fe1080c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.902 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe10dcdb640 1 -- 172.21.15.8:0/329690594 >> v1:172.21.15.8:6789/0 conn(0x7fe1080ca040 legacy=0x7fe10815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.903 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.882+0000 7fe10dcdb640 1 -- 172.21.15.8:0/329690594 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10815dc80 con 0x7fe10815a1f0 2023-10-30T12:44:56.904 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10dcdb640 1 --2- 172.21.15.8:0/329690594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe1080cf030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe100009d90 tx=0x7fe10001f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee871d96176aa82 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:56.905 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe0fffff640 1 -- 172.21.15.8:0/329690594 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000af600 con 0x7fe10815a1f0 2023-10-30T12:44:56.905 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe0fffff640 1 -- 172.21.15.8:0/329690594 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1000ab050 con 0x7fe10815a1f0 2023-10-30T12:44:56.905 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe0fffff640 1 -- 172.21.15.8:0/329690594 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000af980 con 0x7fe10815a1f0 2023-10-30T12:44:56.906 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 -- 172.21.15.8:0/329690594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 msgr2=0x7fe1080cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.906 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 --2- 172.21.15.8:0/329690594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe1080cf030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe100009d90 tx=0x7fe10001f130 comp rx=0 tx=0).stop 2023-10-30T12:44:56.906 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 -- 172.21.15.8:0/329690594 shutdown_connections 2023-10-30T12:44:56.907 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 --2- 172.21.15.8:0/329690594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe1080cf030 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.907 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 --2- 172.21.15.8:0/329690594 >> v2:172.21.15.46:3300/0 conn(0x7fe108153b30 0x7fe1080c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.907 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 -- 172.21.15.8:0/329690594 >> 172.21.15.8:0/329690594 conn(0x7fe10806bfe0 msgr2=0x7fe10806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:56.908 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 -- 172.21.15.8:0/329690594 shutdown_connections 2023-10-30T12:44:56.908 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 -- 172.21.15.8:0/329690594 wait complete. 2023-10-30T12:44:56.908 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.886+0000 7fe10f73e640 1 Processor -- start 2023-10-30T12:44:56.909 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- start start 2023-10-30T12:44:56.909 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 0x7fe1081f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.909 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.910 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe108149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:44:56.910 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108158880 con 0x7fe108153b30 2023-10-30T12:44:56.910 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108154c70 con 0x7fe1080ca040 2023-10-30T12:44:56.911 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1081f2ab0 con 0x7fe10815a1f0 2023-10-30T12:44:56.913 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.913 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10d4da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 0x7fe1081f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.914 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10d4da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 0x7fe1081f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:52868/0 (socket says 172.21.15.8:52868) 2023-10-30T12:44:56.914 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55720/0 (socket says 172.21.15.8:55720) 2023-10-30T12:44:56.914 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10d4da640 1 -- 172.21.15.8:0/1559105942 learned_addr learned my addr 172.21.15.8:0/1559105942 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:44:56.914 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10dcdb640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe108149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:44:56.915 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 -- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 msgr2=0x7fe108149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.915 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe108149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.915 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 -- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 msgr2=0x7fe1081f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.915 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 0x7fe1081f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.915 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 -- 172.21.15.8:0/1559105942 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100009a70 con 0x7fe108153b30 2023-10-30T12:44:56.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10ccd9640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c3c0 tx=0x7fe0f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:44:56.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8005cf0 con 0x7fe108153b30 2023-10-30T12:44:56.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8005e50 con 0x7fe108153b30 2023-10-30T12:44:56.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108151a50 con 0x7fe108153b30 2023-10-30T12:44:56.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80072f0 con 0x7fe108153b30 2023-10-30T12:44:56.917 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe0f8012400 con 0x7fe108153b30 2023-10-30T12:44:56.917 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe108157fe0 con 0x7fe108153b30 2023-10-30T12:44:56.917 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe0f801b400 con 0x7fe108153b30 2023-10-30T12:44:56.917 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.890+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10804e170 con 0x7fe108153b30 2023-10-30T12:44:56.917 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.894+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe0f8017070 con 0x7fe108153b30 2023-10-30T12:44:56.958 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1081f2bf0 con 0x7fe108153b30 2023-10-30T12:44:56.959 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe0fdffb640 1 -- 172.21.15.8:0/1559105942 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe1080caa30 con 0x7fe108153b30 2023-10-30T12:44:56.960 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-30T12:44:56.960 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":10,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:44:55.596742+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:44:56.962 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 msgr2=0x7fe10814f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:44:56.962 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c3c0 tx=0x7fe0f800c880 comp rx=0 tx=0).stop 2023-10-30T12:44:56.962 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 shutdown_connections 2023-10-30T12:44:56.963 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe10815a1f0 0x7fe108149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.963 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe108153b30 0x7fe10814f040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.963 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 --2- 172.21.15.8:0/1559105942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1080ca040 0x7fe1081f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:44:56.964 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 >> 172.21.15.8:0/1559105942 conn(0x7fe10806bfe0 msgr2=0x7fe1080715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:44:56.964 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 shutdown_connections 2023-10-30T12:44:56.964 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:44:56.954+0000 7fe10f73e640 1 -- 172.21.15.8:0/1559105942 wait complete. 2023-10-30T12:44:56.978 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T12:44:56.994 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:56.978+0000 7f0d98eb0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T12:44:57.067 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:57.058+0000 7f0d98eb0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T12:44:57.083 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:57.078+0000 7ffbcd847180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T12:44:57.172 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:57.166+0000 7ffbcd847180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T12:44:57.245 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:57.238+0000 7f0d98eb0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T12:44:57.323 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:57.314+0000 7f0d98eb0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T12:44:57.520 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:57.514+0000 7ffbcd847180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T12:44:57.593 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:57.586+0000 7ffbcd847180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T12:44:57.629 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:57.622+0000 7f0d98eb0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T12:44:57.693 INFO:tasks.ceph.mgr.x.smithi046.stderr:2023-10-30T12:44:57.686+0000 7f0d98eb0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T12:44:58.614 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:58.610+0000 7ffb742e3640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-30T12:44:58.618 INFO:tasks.ceph.mgr.y.smithi008.stderr:2023-10-30T12:44:58.614+0000 7ffb742e3640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-30T12:45:00.067 INFO:tasks.ceph.osd.5.smithi046.stderr:2023-10-30T12:45:00.059+0000 7f68c398c640 -1 osd.5 0 log_to_monitors true 2023-10-30T12:45:00.128 INFO:tasks.ceph.osd.4.smithi046.stderr:2023-10-30T12:45:00.119+0000 7f093fafa640 -1 osd.4 0 log_to_monitors true 2023-10-30T12:45:00.162 INFO:tasks.ceph.osd.7.smithi046.stderr:2023-10-30T12:45:00.155+0000 7fb29427c640 -1 osd.7 0 log_to_monitors true 2023-10-30T12:45:00.174 INFO:tasks.ceph.osd.1.smithi008.stderr:2023-10-30T12:45:00.170+0000 7f3ad7451640 -1 osd.1 0 log_to_monitors true 2023-10-30T12:45:00.189 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-30T12:45:00.182+0000 7ff6c25fd640 -1 osd.0 0 log_to_monitors true 2023-10-30T12:45:00.190 INFO:tasks.ceph.osd.6.smithi046.stderr:2023-10-30T12:45:00.183+0000 7fc67c20d640 -1 osd.6 0 log_to_monitors true 2023-10-30T12:45:00.305 INFO:tasks.ceph.osd.3.smithi008.stderr:2023-10-30T12:45:00.298+0000 7f3e2a6a9640 -1 osd.3 0 log_to_monitors true 2023-10-30T12:45:00.323 INFO:tasks.ceph.osd.2.smithi008.stderr:2023-10-30T12:45:00.318+0000 7f0fe5fb6640 -1 osd.2 0 log_to_monitors true 2023-10-30T12:45:03.357 INFO:tasks.ceph.osd.6.smithi046.stderr:2023-10-30T12:45:03.351+0000 7fc677992640 -1 osd.6 0 waiting for initial osdmap 2023-10-30T12:45:03.363 INFO:tasks.ceph.osd.7.smithi046.stderr:2023-10-30T12:45:03.355+0000 7fb28fa01640 -1 osd.7 0 waiting for initial osdmap 2023-10-30T12:45:03.389 INFO:tasks.ceph.osd.6.smithi046.stderr:2023-10-30T12:45:03.383+0000 7fc66ffd9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.401 INFO:tasks.ceph.osd.7.smithi046.stderr:2023-10-30T12:45:03.395+0000 7fb288048640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.422 INFO:tasks.ceph.osd.5.smithi046.stderr:2023-10-30T12:45:03.415+0000 7f68bf111640 -1 osd.5 0 waiting for initial osdmap 2023-10-30T12:45:03.429 INFO:tasks.ceph.osd.4.smithi046.stderr:2023-10-30T12:45:03.423+0000 7f093b27f640 -1 osd.4 0 waiting for initial osdmap 2023-10-30T12:45:03.441 INFO:tasks.ceph.osd.1.smithi008.stderr:2023-10-30T12:45:03.430+0000 7f3ad2bd6640 -1 osd.1 0 waiting for initial osdmap 2023-10-30T12:45:03.443 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-30T12:45:03.438+0000 7ff6bdd84640 -1 osd.0 0 waiting for initial osdmap 2023-10-30T12:45:03.453 INFO:tasks.ceph.osd.4.smithi046.stderr:2023-10-30T12:45:03.447+0000 7f09338c6640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.453 INFO:tasks.ceph.osd.3.smithi008.stderr:2023-10-30T12:45:03.446+0000 7f3e25e2e640 -1 osd.3 0 waiting for initial osdmap 2023-10-30T12:45:03.457 INFO:tasks.ceph.osd.5.smithi046.stderr:2023-10-30T12:45:03.451+0000 7f68b7758640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.471 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-30T12:45:03.466+0000 7ff6b63cb640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.474 INFO:tasks.ceph.osd.2.smithi008.stderr:2023-10-30T12:45:03.470+0000 7f0fe173b640 -1 osd.2 0 waiting for initial osdmap 2023-10-30T12:45:03.479 INFO:tasks.ceph.osd.1.smithi008.stderr:2023-10-30T12:45:03.474+0000 7f3acb21d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.486 INFO:tasks.ceph.osd.3.smithi008.stderr:2023-10-30T12:45:03.482+0000 7f3e1e475640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.507 INFO:tasks.ceph.osd.2.smithi008.stderr:2023-10-30T12:45:03.502+0000 7f0fd9d82640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T12:45:03.784 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:03.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.910+0000 7f047b017640 1 Processor -- start 2023-10-30T12:45:03.916 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.910+0000 7f047b017640 1 -- start start 2023-10-30T12:45:03.918 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.910+0000 7f047b017640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0474153a90 0x7f047415a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.918 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.910+0000 7f047b017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f047406f8e0 0x7f047406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.918 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.910+0000 7f047b017640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047406ed50 con 0x7f0474153e70 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f047b017640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047406ce80 con 0x7f0474153a90 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f047b017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740c8060 con 0x7f047406f8e0 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0474153a90 0x7f047415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0473fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0474153e70 0x7f04740c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:49890/0 (socket says 172.21.15.8:49890) 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0473fff640 1 -- 172.21.15.8:0/466465728 learned_addr learned my addr 172.21.15.8:0/466465728 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:03.919 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f04795b4640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f047406f8e0 0x7f047406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.920 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 -- 172.21.15.8:0/466465728 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f047406f8e0 msgr2=0x7f047406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:03.920 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f047406f8e0 0x7f047406fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.920 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 -- 172.21.15.8:0/466465728 >> v1:172.21.15.8:6789/0 conn(0x7f0474153e70 legacy=0x7f04740c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.920 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 -- 172.21.15.8:0/466465728 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047415a8d0 con 0x7f0474153a90 2023-10-30T12:45:03.920 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f04737fe640 1 -- 172.21.15.8:0/466465728 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442605613 0 0) 0x7f047406ed50 con 0x7f0474153e70 2023-10-30T12:45:03.921 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f0478db3640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f047415a390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04680097c0 tx=0x7f046801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788f3c6ea55608ff server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.921 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f04737fe640 1 -- 172.21.15.8:0/466465728 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680ac620 con 0x7f0474153a90 2023-10-30T12:45:03.921 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f04737fe640 1 -- 172.21.15.8:0/466465728 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04680ac780 con 0x7f0474153a90 2023-10-30T12:45:03.922 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.914+0000 7f04737fe640 1 -- 172.21.15.8:0/466465728 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680acb10 con 0x7f0474153a90 2023-10-30T12:45:03.922 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- 172.21.15.8:0/466465728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 msgr2=0x7f047415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f047415a390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04680097c0 tx=0x7f046801c3c0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- 172.21.15.8:0/466465728 shutdown_connections 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f047406f8e0 0x7f047406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- 172.21.15.8:0/466465728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f047415a390 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- 172.21.15.8:0/466465728 >> 172.21.15.8:0/466465728 conn(0x7f04740ccf00 msgr2=0x7f04740cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:03.923 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- 172.21.15.8:0/466465728 shutdown_connections 2023-10-30T12:45:03.924 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- 172.21.15.8:0/466465728 wait complete. 2023-10-30T12:45:03.925 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 Processor -- start 2023-10-30T12:45:03.925 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- start start 2023-10-30T12:45:03.926 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 0x7f04741f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.926 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.926 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0474153e70 0x7f047414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.926 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741f5bf0 con 0x7f0474153a90 2023-10-30T12:45:03.926 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474152c70 con 0x7f047406f8e0 2023-10-30T12:45:03.927 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f047b017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04741f4c60 con 0x7f0474153e70 2023-10-30T12:45:03.927 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.927 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55814/0 (socket says 172.21.15.8:55814) 2023-10-30T12:45:03.927 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 -- 172.21.15.8:0/1976445430 learned_addr learned my addr 172.21.15.8:0/1976445430 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:03.927 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f04795b4640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0474153e70 0x7f047414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.929 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0478db3640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 0x7f04741f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.929 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 -- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0474153e70 msgr2=0x7f047414a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.929 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0474153e70 0x7f047414a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.929 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 -- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 msgr2=0x7f04741f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:03.930 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 0x7f04741f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:03.930 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.918+0000 7f0473fff640 1 -- 172.21.15.8:0/1976445430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009420 con 0x7f0474153a90 2023-10-30T12:45:03.930 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f0478db3640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 0x7f04741f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:03.930 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f0473fff640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f046000c4b0 tx=0x7f046000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.931 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600074e0 con 0x7f0474153a90 2023-10-30T12:45:03.932 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0460007660 con 0x7f0474153a90 2023-10-30T12:45:03.932 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047415c130 con 0x7f0474153a90 2023-10-30T12:45:03.932 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460004a30 con 0x7f0474153a90 2023-10-30T12:45:03.932 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04741f2210 con 0x7f0474153a90 2023-10-30T12:45:03.932 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047404e170 con 0x7f0474153a90 2023-10-30T12:45:03.933 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 85794+0+0 (secure 0 0 0) 0x7f0460004b90 con 0x7f0474153a90 2023-10-30T12:45:03.933 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 0x7f04580688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:03.933 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f04600821d0 con 0x7f0474153a90 2023-10-30T12:45:03.933 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.922+0000 7f0478db3640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 0x7f04580688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:03.933 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.926+0000 7f0478db3640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 0x7f04580688a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04680ae000 tx=0x7f04680a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:03.934 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:03.930+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f04600504f0 con 0x7f0474153a90 2023-10-30T12:45:04.092 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.086+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f047414e240 con 0x7f0474153a90 2023-10-30T12:45:04.092 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.086+0000 7f04717fa640 1 -- 172.21.15.8:0/1976445430 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0460018070 con 0x7f0474153a90 2023-10-30T12:45:04.093 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-30T12:45:04.093 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":12,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:01.615910+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:04.095 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 msgr2=0x7f04580688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:04.095 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 0x7f04580688a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04680ae000 tx=0x7f04680a7000 comp rx=0 tx=0).stop 2023-10-30T12:45:04.095 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 msgr2=0x7f04740c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:04.095 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f046000c4b0 tx=0x7f046000c970 comp rx=0 tx=0).stop 2023-10-30T12:45:04.096 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 shutdown_connections 2023-10-30T12:45:04.096 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04580663e0 0x7f04580688a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:04.096 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0474153e70 0x7f047414a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:04.096 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0474153a90 0x7f04740c9c60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:04.096 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 --2- 172.21.15.8:0/1976445430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f047406f8e0 0x7f04741f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:04.097 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 >> 172.21.15.8:0/1976445430 conn(0x7f04740ccf00 msgr2=0x7f04740c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:04.097 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 shutdown_connections 2023-10-30T12:45:04.097 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:04.090+0000 7f047b017640 1 -- 172.21.15.8:0/1976445430 wait complete. 2023-10-30T12:45:04.112 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T12:45:10.918 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:11.061 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.054+0000 7f1b9f3c4640 1 Processor -- start 2023-10-30T12:45:11.062 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.054+0000 7f1b9f3c4640 1 -- start start 2023-10-30T12:45:11.063 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9f3c4640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1b98150820 0x7f1b980c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.063 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9f3c4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b980703c0 0x7f1b98156440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.063 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9f3c4640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b98150c00 con 0x7f1b980c7b10 2023-10-30T12:45:11.064 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9f3c4640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980717f0 con 0x7f1b98150820 2023-10-30T12:45:11.064 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9f3c4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980c7f30 con 0x7f1b980703c0 2023-10-30T12:45:11.064 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1b98150820 0x7f1b980c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.064 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1b98150820 0x7f1b980c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34248/0 (socket says 172.21.15.8:34248) 2023-10-30T12:45:11.064 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 -- 172.21.15.8:0/1959867064 learned_addr learned my addr 172.21.15.8:0/1959867064 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:11.065 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9c95f640 1 --1- 172.21.15.8:0/1959867064 >> v1:172.21.15.8:6789/0 conn(0x7f1b980c7b10 0x7f1b9806fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43570/0 (socket says 172.21.15.8:43570) 2023-10-30T12:45:11.065 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d961640 1 --2- 172.21.15.8:0/1959867064 >> v2:172.21.15.46:3300/0 conn(0x7f1b980703c0 0x7f1b98156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.065 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 -- 172.21.15.8:0/1959867064 >> v2:172.21.15.46:3300/0 conn(0x7f1b980703c0 msgr2=0x7f1b98156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.066 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 --2- 172.21.15.8:0/1959867064 >> v2:172.21.15.46:3300/0 conn(0x7f1b980703c0 0x7f1b98156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.066 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 -- 172.21.15.8:0/1959867064 >> v1:172.21.15.8:6789/0 conn(0x7f1b980c7b10 legacy=0x7f1b9806fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.066 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b9d160640 1 -- 172.21.15.8:0/1959867064 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980c6f90 con 0x7f1b98150820 2023-10-30T12:45:11.066 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.058+0000 7f1b8ffff640 1 -- 172.21.15.8:0/1959867064 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689479263 0 0) 0x7f1b98150c00 con 0x7f1b980c7b10 2023-10-30T12:45:11.068 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9d160640 1 --2- 172.21.15.8:0/1959867064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b98150820 0x7f1b980c75d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b9806d120 tx=0x7f1b9401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a51b915ad578813 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.068 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b8ffff640 1 -- 172.21.15.8:0/1959867064 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940a89f0 con 0x7f1b98150820 2023-10-30T12:45:11.068 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b8ffff640 1 -- 172.21.15.8:0/1959867064 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b940a8b50 con 0x7f1b98150820 2023-10-30T12:45:11.068 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b8ffff640 1 -- 172.21.15.8:0/1959867064 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940b0440 con 0x7f1b98150820 2023-10-30T12:45:11.069 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/1959867064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b98150820 msgr2=0x7f1b980c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.069 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/1959867064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b98150820 0x7f1b980c75d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b9806d120 tx=0x7f1b9401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.069 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/1959867064 shutdown_connections 2023-10-30T12:45:11.069 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/1959867064 >> v2:172.21.15.46:3300/0 conn(0x7f1b980703c0 0x7f1b98156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.069 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/1959867064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b98150820 0x7f1b980c75d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.070 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/1959867064 >> 172.21.15.8:0/1959867064 conn(0x7f1b9806b5c0 msgr2=0x7f1b9806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.070 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/1959867064 shutdown_connections 2023-10-30T12:45:11.070 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/1959867064 wait complete. 2023-10-30T12:45:11.073 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 Processor -- start 2023-10-30T12:45:11.073 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.062+0000 7f1b9f3c4640 1 -- start start 2023-10-30T12:45:11.073 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1b980703c0 0x7f1b980c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.073 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.074 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b98150820 0x7f1b981454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.074 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981eebe0 con 0x7f1b980c7b10 2023-10-30T12:45:11.074 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981efbb0 con 0x7f1b98150820 2023-10-30T12:45:11.074 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980c2ed0 con 0x7f1b980703c0 2023-10-30T12:45:11.075 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.075 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34252/0 (socket says 172.21.15.8:34252) 2023-10-30T12:45:11.075 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 -- 172.21.15.8:0/3955542830 learned_addr learned my addr 172.21.15.8:0/3955542830 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:11.075 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9d961640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b98150820 0x7f1b981454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.075 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9d160640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1b980703c0 0x7f1b980c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.076 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 -- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1b980703c0 msgr2=0x7f1b980c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.076 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1b980703c0 0x7f1b980c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.076 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 -- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b98150820 msgr2=0x7f1b981454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.076 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b98150820 0x7f1b981454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.076 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 -- 172.21.15.8:0/3955542830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940094d0 con 0x7f1b980c7b10 2023-10-30T12:45:11.077 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9c95f640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b880011f0 tx=0x7f1b8800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.077 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880074e0 con 0x7f1b980c7b10 2023-10-30T12:45:11.077 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88007640 con 0x7f1b980c7b10 2023-10-30T12:45:11.077 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88004a10 con 0x7f1b980c7b10 2023-10-30T12:45:11.077 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b980c2bf0 con 0x7f1b980c7b10 2023-10-30T12:45:11.078 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981ef0d0 con 0x7f1b980c7b10 2023-10-30T12:45:11.078 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.066+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9804e170 con 0x7f1b980c7b10 2023-10-30T12:45:11.079 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.070+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b88004b70 con 0x7f1b980c7b10 2023-10-30T12:45:11.079 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.070+0000 7f1b8dffb640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 0x7f1b7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.079 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.070+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b88082e40 con 0x7f1b980c7b10 2023-10-30T12:45:11.079 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.070+0000 7f1b9d160640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 0x7f1b7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.080 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.070+0000 7f1b9d160640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 0x7f1b7c068960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b940b2000 tx=0x7f1b94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.080 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.074+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b88050570 con 0x7f1b980c7b10 2023-10-30T12:45:11.238 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.230+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b980c4ff0 con 0x7f1b980c7b10 2023-10-30T12:45:11.238 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b8dffb640 1 -- 172.21.15.8:0/3955542830 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12730 (secure 0 0 0) 0x7f1b88018070 con 0x7f1b980c7b10 2023-10-30T12:45:11.239 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-30T12:45:11.239 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":16,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:08.010934+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:11.241 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 msgr2=0x7f1b7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.241 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 0x7f1b7c068960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b940b2000 tx=0x7f1b94008040 comp rx=0 tx=0).stop 2023-10-30T12:45:11.241 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 msgr2=0x7f1b981542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.241 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b880011f0 tx=0x7f1b8800c9b0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 shutdown_connections 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1b7c0664a0 0x7f1b7c068960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b98150820 0x7f1b981454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b980c7b10 0x7f1b981542c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 --2- 172.21.15.8:0/3955542830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1b980703c0 0x7f1b980c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.242 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 >> 172.21.15.8:0/3955542830 conn(0x7f1b9806b5c0 msgr2=0x7f1b980afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.243 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 shutdown_connections 2023-10-30T12:45:11.243 INFO:teuthology.misc.health.smithi008.stderr:2023-10-30T12:45:11.234+0000 7f1b9f3c4640 1 -- 172.21.15.8:0/3955542830 wait complete. 2023-10-30T12:45:11.259 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-30T12:45:11.259 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-30T12:45:11.260 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-30T12:45:11.260 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-30T12:45:11.264 INFO:tasks.ceph.mds.a:Started 2023-10-30T12:45:11.264 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:45:11.265 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:45:11.268 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:45:11.268 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:45:11.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:45:11.272 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:45:11.272 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-30T12:45:11.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-30T12:45:11.275 INFO:tasks.ceph.mds.b:Started 2023-10-30T12:45:11.276 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-30T12:45:11.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-30T12:45:11.279 INFO:tasks.ceph.mds.d:Started 2023-10-30T12:45:11.279 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-30T12:45:11.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-30T12:45:11.282 INFO:tasks.ceph.mds.f:Started 2023-10-30T12:45:11.282 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-30T12:45:11.283 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:11.356 INFO:tasks.ceph.mds.f.smithi046.stdout:starting mds.f at 2023-10-30T12:45:11.364 INFO:tasks.ceph.mds.d.smithi046.stdout:starting mds.d at 2023-10-30T12:45:11.364 INFO:tasks.ceph.mds.b.smithi046.stdout:starting mds.b at 2023-10-30T12:45:11.366 INFO:tasks.ceph.mds.a.smithi008.stdout:starting mds.a at 2023-10-30T12:45:11.367 INFO:tasks.ceph.mds.e.smithi008.stdout:starting mds.e at 2023-10-30T12:45:11.367 INFO:tasks.ceph.mds.c.smithi008.stdout:starting mds.c at 2023-10-30T12:45:11.368 INFO:tasks.ceph.mds.e.smithi008.stderr:2023-10-30T12:45:11.362+0000 7f41c0c37640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-30T12:45:11.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.430+0000 7f9e82091640 1 Processor -- start 2023-10-30T12:45:11.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.430+0000 7f9e82091640 1 -- start start 2023-10-30T12:45:11.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.430+0000 7f9e82091640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e82091640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e7c0c3e40 0x7f9e7c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e82091640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7c0c2310 con 0x7f9e7c150830 2023-10-30T12:45:11.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e82091640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c06bc10 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e82091640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c156710 con 0x7f9e7c0c3e40 2023-10-30T12:45:11.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34296/0 (socket says 172.21.15.8:34296) 2023-10-30T12:45:11.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 -- 172.21.15.8:0/3271969295 learned_addr learned my addr 172.21.15.8:0/3271969295 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:11.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7bfff640 1 --2- 172.21.15.8:0/3271969295 >> v2:172.21.15.46:3300/0 conn(0x7f9e7c0c3e40 0x7f9e7c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 -- 172.21.15.8:0/3271969295 >> v2:172.21.15.46:3300/0 conn(0x7f9e7c0c3e40 msgr2=0x7f9e7c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 --2- 172.21.15.8:0/3271969295 >> v2:172.21.15.46:3300/0 conn(0x7f9e7c0c3e40 0x7f9e7c070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 -- 172.21.15.8:0/3271969295 >> v1:172.21.15.46:6789/0 conn(0x7f9e7c150830 legacy=0x7f9e7c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 -- 172.21.15.8:0/3271969295 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c0c4360 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7a7fc640 1 -- 172.21.15.8:0/3271969295 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787814142 0 0) 0x7f9e7c0c2310 con 0x7f9e7c150830 2023-10-30T12:45:11.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7affd640 1 --2- 172.21.15.8:0/3271969295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c066e10 tx=0x7f9e7001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739b0e2843129009 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7a7fc640 1 -- 172.21.15.8:0/3271969295 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700a89f0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7a7fc640 1 -- 172.21.15.8:0/3271969295 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700a8b50 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.434+0000 7f9e7a7fc640 1 -- 172.21.15.8:0/3271969295 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700b0440 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- 172.21.15.8:0/3271969295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 msgr2=0x7f9e7c0c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- 172.21.15.8:0/3271969295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c066e10 tx=0x7f9e7001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- 172.21.15.8:0/3271969295 shutdown_connections 2023-10-30T12:45:11.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- 172.21.15.8:0/3271969295 >> v2:172.21.15.46:3300/0 conn(0x7f9e7c0c3e40 0x7f9e7c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- 172.21.15.8:0/3271969295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c0c3900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- 172.21.15.8:0/3271969295 >> 172.21.15.8:0/3271969295 conn(0x7f9e7c0cac70 msgr2=0x7f9e7c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- 172.21.15.8:0/3271969295 shutdown_connections 2023-10-30T12:45:11.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- 172.21.15.8:0/3271969295 wait complete. 2023-10-30T12:45:11.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 Processor -- start 2023-10-30T12:45:11.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- start start 2023-10-30T12:45:11.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 0x7f9e7c151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e7c150830 0x7f9e7c1efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c158bf0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c158e70 con 0x7f9e7c0c3e40 2023-10-30T12:45:11.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e82091640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c159150 con 0x7f9e7c150830 2023-10-30T12:45:11.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e7affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 0x7f9e7c151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e7affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34300/0 (socket says 172.21.15.8:34300) 2023-10-30T12:45:11.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e7affd640 1 -- 172.21.15.8:0/717588834 learned_addr learned my addr 172.21.15.8:0/717588834 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:11.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.438+0000 7f9e7bfff640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e7c150830 0x7f9e7c1efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 -- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e7c150830 msgr2=0x7f9e7c1efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e7c150830 0x7f9e7c1efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 -- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 msgr2=0x7f9e7c151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 0x7f9e7c151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 -- 172.21.15.8:0/717588834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700094d0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7b7fe640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 0x7f9e7c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:11.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7affd640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001b900 tx=0x7f9e7001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700b1ce0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700b1e40 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700bb670 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c159430 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c1599f0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c04e170 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e700b7070 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 0x7f9e68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e70121ac0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.442+0000 7f9e7b7fe640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 0x7f9e68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.446+0000 7f9e7b7fe640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 0x7f9e68068960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000f70 tx=0x7f9e6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.450+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e700ef2a0 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.606+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e7c15aa00 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.606+0000 7f9e63fff640 1 -- 172.21.15.8:0/717588834 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12730 (secure 0 0 0) 0x7f9e700b5090 con 0x7f9e7c0cb7d0 2023-10-30T12:45:11.613 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:11.613 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":16,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:08.010934+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:11.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 msgr2=0x7f9e68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 0x7f9e68068960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000f70 tx=0x7f9e6c009040 comp rx=0 tx=0).stop 2023-10-30T12:45:11.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 msgr2=0x7f9e7c1538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001b900 tx=0x7f9e7001ba90 comp rx=0 tx=0).stop 2023-10-30T12:45:11.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 shutdown_connections 2023-10-30T12:45:11.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e680664a0 0x7f9e68068960 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e7c150830 0x7f9e7c1efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e7c0cb7d0 0x7f9e7c1538d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 --2- 172.21.15.8:0/717588834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e7c0c3e40 0x7f9e7c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 >> 172.21.15.8:0/717588834 conn(0x7f9e7c0cac70 msgr2=0x7f9e7c0733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 shutdown_connections 2023-10-30T12:45:11.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.610+0000 7f9e82091640 1 -- 172.21.15.8:0/717588834 wait complete. 2023-10-30T12:45:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:11.635 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:11.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.774+0000 7f09456ad640 1 Processor -- start 2023-10-30T12:45:11.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.774+0000 7f09456ad640 1 -- start start 2023-10-30T12:45:11.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f09456ad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09400718d0 0x7f09400c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f09456ad640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f0940153750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f09456ad640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0940153c90 con 0x7f0940071cb0 2023-10-30T12:45:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f09456ad640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09400cca10 con 0x7f09400718d0 2023-10-30T12:45:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f09456ad640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094015d0a0 con 0x7f0940153300 2023-10-30T12:45:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093f7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0940071cb0 0x7f094015a230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43598/0 (socket says 172.21.15.8:43598) 2023-10-30T12:45:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093f7fe640 1 -- 172.21.15.8:0/3640872965 learned_addr learned my addr 172.21.15.8:0/3640872965 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:11.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093ffff640 1 --2- 172.21.15.8:0/3640872965 >> v2:172.21.15.46:3300/0 conn(0x7f09400718d0 0x7f09400c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f0940153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587265189 0 0) 0x7f0940153c90 con 0x7f0940071cb0 2023-10-30T12:45:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0928003530 con 0x7f0940071cb0 2023-10-30T12:45:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529061900 0 0) 0x7f0928003530 con 0x7f0940071cb0 2023-10-30T12:45:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 msgr2=0x7f0940153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f0940153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 >> v2:172.21.15.46:3300/0 conn(0x7f09400718d0 msgr2=0x7f09400c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 --2- 172.21.15.8:0/3640872965 >> v2:172.21.15.46:3300/0 conn(0x7f09400718d0 0x7f09400c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094015d380 con 0x7f0940071cb0 2023-10-30T12:45:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f0934001680 con 0x7f0940071cb0 2023-10-30T12:45:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 >> v1:172.21.15.8:6789/0 conn(0x7f0940071cb0 legacy=0x7f094015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0928003980 0x7f0928003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f09280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0928004d10 0x7f0928005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400cca10 con 0x7f0928004d10 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094015d0a0 con 0x7f0928003980 2023-10-30T12:45:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092800b990 con 0x7f0940153300 2023-10-30T12:45:11.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0928004d10 0x7f0928005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093ffff640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0928003980 0x7f0928003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f093f7fe640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f09280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 -- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 msgr2=0x7f09280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0940153300 0x7f09280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 -- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0928003980 msgr2=0x7f0928003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:11.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0928003980 0x7f0928003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.778+0000 7f0944eac640 1 -- 172.21.15.8:0/3640872965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928003530 con 0x7f0928004d10 2023-10-30T12:45:11.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f0944eac640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0928004d10 0x7f0928005190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0938009fc0 tx=0x7f093801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e60ce1d91a771d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380a5c70 con 0x7f0928004d10 2023-10-30T12:45:11.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093800f040 con 0x7f0928004d10 2023-10-30T12:45:11.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f093effd640 1 -- 172.21.15.8:0/3640872965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380ad440 con 0x7f0928004d10 2023-10-30T12:45:11.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- 172.21.15.8:0/3640872965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0928004d10 msgr2=0x7f0928005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 --2- 172.21.15.8:0/3640872965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0928004d10 0x7f0928005190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0938009fc0 tx=0x7f093801cc60 comp rx=0 tx=0).stop 2023-10-30T12:45:11.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f0944eac640 1 -- 172.21.15.8:0/3640872965 reap_dead start 2023-10-30T12:45:11.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- 172.21.15.8:0/3640872965 shutdown_connections 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- 172.21.15.8:0/3640872965 >> 172.21.15.8:0/3640872965 conn(0x7f094006ab00 msgr2=0x7f0940065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- 172.21.15.8:0/3640872965 shutdown_connections 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- 172.21.15.8:0/3640872965 wait complete. 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 Processor -- start 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.782+0000 7f09456ad640 1 -- start start 2023-10-30T12:45:11.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0928004d10 0x7f09400c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 0x7f09401ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0940158860 con 0x7f094014c100 2023-10-30T12:45:11.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0940158ae0 con 0x7f0940157500 2023-10-30T12:45:11.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09401592b0 con 0x7f0928004d10 2023-10-30T12:45:11.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34318/0 (socket says 172.21.15.8:34318) 2023-10-30T12:45:11.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 -- 172.21.15.8:0/3699571171 learned_addr learned my addr 172.21.15.8:0/3699571171 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:11.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093ffff640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0928004d10 0x7f09400c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093f7fe640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 0x7f09401ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 -- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0928004d10 msgr2=0x7f09400c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0928004d10 0x7f09400c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 -- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 msgr2=0x7f09401ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 0x7f09401ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 -- 172.21.15.8:0/3699571171 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400c6b80 con 0x7f094014c100 2023-10-30T12:45:11.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093f7fe640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 0x7f09401ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:11.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f0944eac640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f093801c6d0 tx=0x7f0938009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380039f0 con 0x7f094014c100 2023-10-30T12:45:11.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938003b70 con 0x7f094014c100 2023-10-30T12:45:11.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380ae890 con 0x7f094014c100 2023-10-30T12:45:11.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09401ef550 con 0x7f094014c100 2023-10-30T12:45:11.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0940157940 con 0x7f094014c100 2023-10-30T12:45:11.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.786+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094004e170 con 0x7f094014c100 2023-10-30T12:45:11.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.790+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09380b4040 con 0x7f094014c100 2023-10-30T12:45:11.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.790+0000 7f093cff9640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 0x7f092c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:11.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.790+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f093811db40 con 0x7f094014c100 2023-10-30T12:45:11.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.790+0000 7f093ffff640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 0x7f092c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:11.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.790+0000 7f093ffff640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 0x7f092c068960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0930009bc0 tx=0x7f0930009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:11.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.794+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f09380eb200 con 0x7f094014c100 2023-10-30T12:45:11.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.950+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0940152c00 con 0x7f094014c100 2023-10-30T12:45:11.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.950+0000 7f093cff9640 1 -- 172.21.15.8:0/3699571171 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09380b2070 con 0x7f094014c100 2023-10-30T12:45:11.957 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:11.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 msgr2=0x7f092c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 0x7f092c068960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0930009bc0 tx=0x7f0930009190 comp rx=0 tx=0).stop 2023-10-30T12:45:11.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 msgr2=0x7f09400b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:11.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f093801c6d0 tx=0x7f0938009750 comp rx=0 tx=0).stop 2023-10-30T12:45:11.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 shutdown_connections 2023-10-30T12:45:11.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f092c0664a0 0x7f092c068960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0940157500 0x7f09401ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094014c100 0x7f09400b35a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 --2- 172.21.15.8:0/3699571171 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0928004d10 0x7f09400c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:11.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 >> 172.21.15.8:0/3699571171 conn(0x7f094006ab00 msgr2=0x7f09400733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:11.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 shutdown_connections 2023-10-30T12:45:11.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:11.954+0000 7f09456ad640 1 -- 172.21.15.8:0/3699571171 wait complete. 2023-10-30T12:45:11.977 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 Processor -- start 2023-10-30T12:45:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 -- start start 2023-10-30T12:45:12.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 0x7f25c015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25c00c42a0 0x7f25c00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c0159f20 con 0x7f25c0153c80 2023-10-30T12:45:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c015dec0 con 0x7f25c00c42a0 2023-10-30T12:45:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c832a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c01599a0 con 0x7f25c00c4a10 2023-10-30T12:45:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25c00c42a0 0x7f25c00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25c00c42a0 0x7f25c00ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34332/0 (socket says 172.21.15.8:34332) 2023-10-30T12:45:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 -- 172.21.15.8:0/1803240904 learned_addr learned my addr 172.21.15.8:0/1803240904 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c58c5640 1 --2- 172.21.15.8:0/1803240904 >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 0x7f25c015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 -- 172.21.15.8:0/1803240904 >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 msgr2=0x7f25c015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 --2- 172.21.15.8:0/1803240904 >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 0x7f25c015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 -- 172.21.15.8:0/1803240904 >> v1:172.21.15.46:6789/0 conn(0x7f25c0153c80 legacy=0x7f25c015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c68c7640 1 -- 172.21.15.8:0/1803240904 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c015d620 con 0x7f25c00c42a0 2023-10-30T12:45:12.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.118+0000 7f25c58c5640 1 --2- 172.21.15.8:0/1803240904 >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 0x7f25c015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:12.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c68c7640 1 --2- 172.21.15.8:0/1803240904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c00ca1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25c006be50 tx=0x7f25b801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d7a680188aea1c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c50c4640 1 -- 172.21.15.8:0/1803240904 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80af400 con 0x7f25c00c42a0 2023-10-30T12:45:12.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c50c4640 1 -- 172.21.15.8:0/1803240904 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b80ab050 con 0x7f25c00c42a0 2023-10-30T12:45:12.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c50c4640 1 -- 172.21.15.8:0/1803240904 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80af780 con 0x7f25c00c42a0 2023-10-30T12:45:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 -- 172.21.15.8:0/1803240904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 msgr2=0x7f25c00ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 --2- 172.21.15.8:0/1803240904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c00ca1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25c006be50 tx=0x7f25b801f1c0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 -- 172.21.15.8:0/1803240904 shutdown_connections 2023-10-30T12:45:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 --2- 172.21.15.8:0/1803240904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c00ca1e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 --2- 172.21.15.8:0/1803240904 >> v2:172.21.15.46:3300/0 conn(0x7f25c00c4a10 0x7f25c015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 -- 172.21.15.8:0/1803240904 >> 172.21.15.8:0/1803240904 conn(0x7f25c00715c0 msgr2=0x7f25c00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 -- 172.21.15.8:0/1803240904 shutdown_connections 2023-10-30T12:45:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.122+0000 7f25c832a640 1 -- 172.21.15.8:0/1803240904 wait complete. 2023-10-30T12:45:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 Processor -- start 2023-10-30T12:45:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- start start 2023-10-30T12:45:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 0x7f25c00ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25c0153c80 0x7f25c00ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01f2f80 con 0x7f25c00c42a0 2023-10-30T12:45:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01f2ca0 con 0x7f25c00c4a10 2023-10-30T12:45:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00cfa30 con 0x7f25c0153c80 2023-10-30T12:45:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34338/0 (socket says 172.21.15.8:34338) 2023-10-30T12:45:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 -- 172.21.15.8:0/3615736414 learned_addr learned my addr 172.21.15.8:0/3615736414 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c58c5640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 0x7f25c00ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c68c7640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25c0153c80 0x7f25c00ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 -- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25c0153c80 msgr2=0x7f25c00ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25c0153c80 0x7f25c00ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 -- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 msgr2=0x7f25c00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 0x7f25c00ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 -- 172.21.15.8:0/3615736414 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8009b30 con 0x7f25c00c42a0 2023-10-30T12:45:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c58c5640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 0x7f25c00ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c60c6640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c370 tx=0x7f25bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0052d0 con 0x7f25c00c42a0 2023-10-30T12:45:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc005430 con 0x7f25c00c42a0 2023-10-30T12:45:12.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c00cfd10 con 0x7f25c00c42a0 2023-10-30T12:45:12.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc007260 con 0x7f25c00c42a0 2023-10-30T12:45:12.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00cdae0 con 0x7f25c00c42a0 2023-10-30T12:45:12.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.126+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c004e170 con 0x7f25c00c42a0 2023-10-30T12:45:12.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.130+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25bc0073c0 con 0x7f25c00c42a0 2023-10-30T12:45:12.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.130+0000 7f25b6ffd640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 0x7f25a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:12.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.130+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25bc082fe0 con 0x7f25c00c42a0 2023-10-30T12:45:12.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.130+0000 7f25c58c5640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 0x7f25a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:12.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.130+0000 7f25c58c5640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 0x7f25a4068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25c00cbb20 tx=0x7f25b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:12.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.134+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f25bc050680 con 0x7f25c00c42a0 2023-10-30T12:45:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.322+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c00cdcd0 con 0x7f25c00c42a0 2023-10-30T12:45:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.322+0000 7f25b6ffd640 1 -- 172.21.15.8:0/3615736414 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f25bc053ca0 con 0x7f25c00c42a0 2023-10-30T12:45:12.330 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:12.330 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-30T12:45:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 2 2023-10-30T12:45:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 msgr2=0x7f25a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 0x7f25a4068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25c00cbb20 tx=0x7f25b0008040 comp rx=0 tx=0).stop 2023-10-30T12:45:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 msgr2=0x7f25c01f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c370 tx=0x7f25bc00c830 comp rx=0 tx=0).stop 2023-10-30T12:45:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 shutdown_connections 2023-10-30T12:45:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25a40664a0 0x7f25a4068960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c00c4a10 0x7f25c00ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25c0153c80 0x7f25c00ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 --2- 172.21.15.8:0/3615736414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00c42a0 0x7f25c01f2240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:12.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 >> 172.21.15.8:0/3615736414 conn(0x7f25c00715c0 msgr2=0x7f25c0151450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:12.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 shutdown_connections 2023-10-30T12:45:12.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:12.326+0000 7f25c832a640 1 -- 172.21.15.8:0/3615736414 wait complete. 2023-10-30T12:45:13.875 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:13.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:14.351 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:14.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.498+0000 7f8f4d69c640 1 Processor -- start 2023-10-30T12:45:14.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.498+0000 7f8f4d69c640 1 -- start start 2023-10-30T12:45:14.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f4d69c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f48153c80 0x7f8f4815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f4d69c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f48159f20 con 0x7f8f480c4a10 2023-10-30T12:45:14.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f4d69c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4815d620 con 0x7f8f480c42a0 2023-10-30T12:45:14.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f4d69c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4815dec0 con 0x7f8f48153c80 2023-10-30T12:45:14.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f3ffff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8f480c4a10 0x7f8f4815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43612/0 (socket says 172.21.15.8:43612) 2023-10-30T12:45:14.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f3ffff640 1 -- 172.21.15.8:0/3785075310 learned_addr learned my addr 172.21.15.8:0/3785075310 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:14.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f48153c80 0x7f8f4815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339044085 0 0) 0x7f8f48159f20 con 0x7f8f480c4a10 2023-10-30T12:45:14.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f34003530 con 0x7f8f480c4a10 2023-10-30T12:45:14.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773729728 0 0) 0x7f8f34003530 con 0x7f8f480c4a10 2023-10-30T12:45:14.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f48153c80 msgr2=0x7f8f4815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f48153c80 0x7f8f4815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 >> v1:172.21.15.46:6789/0 conn(0x7f8f480c42a0 legacy=0x7f8f480ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4815d900 con 0x7f8f480c4a10 2023-10-30T12:45:14.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f8f30001680 con 0x7f8f480c4a10 2023-10-30T12:45:14.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 >> v1:172.21.15.8:6789/0 conn(0x7f8f480c4a10 legacy=0x7f8f4815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f340039b0 0x7f8f34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f34004360 0x7f8f34004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f34005030 0x7f8f3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4815dec0 con 0x7f8f340039b0 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3400bed0 con 0x7f8f34004360 2023-10-30T12:45:14.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3400c010 con 0x7f8f34005030 2023-10-30T12:45:14.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f48153c80 0x7f8f4815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:14.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f340039b0 0x7f8f34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f477fe640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f34005030 0x7f8f3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f3ffff640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f34004360 0x7f8f34004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 -- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f34005030 msgr2=0x7f8f3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f34005030 0x7f8f3400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 -- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f34004360 msgr2=0x7f8f34004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f34004360 0x7f8f34004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f46ffd640 1 -- 172.21.15.8:0/3785075310 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34003530 con 0x7f8f340039b0 2023-10-30T12:45:14.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.502+0000 7f8f477fe640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f34005030 0x7f8f3400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:14.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f340039b0 0x7f8f34003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cf190 tx=0x7f8f3801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58290f9a7f04b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004030 con 0x7f8f340039b0 2023-10-30T12:45:14.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38004190 con 0x7f8f340039b0 2023-10-30T12:45:14.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f467fc640 1 -- 172.21.15.8:0/3785075310 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004520 con 0x7f8f340039b0 2023-10-30T12:45:14.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3785075310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f340039b0 msgr2=0x7f8f34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3785075310 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f340039b0 0x7f8f34003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cf190 tx=0x7f8f3801ca90 comp rx=0 tx=0).stop 2023-10-30T12:45:14.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f477fe640 1 -- 172.21.15.8:0/3785075310 reap_dead start 2023-10-30T12:45:14.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3785075310 shutdown_connections 2023-10-30T12:45:14.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3785075310 >> 172.21.15.8:0/3785075310 conn(0x7f8f480715c0 msgr2=0x7f8f480719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:14.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3785075310 shutdown_connections 2023-10-30T12:45:14.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.506+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3785075310 wait complete. 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 Processor -- start 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- start start 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 0x7f8f481ef040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4815b910 0x7f8f480c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48154b70 con 0x7f8f4815bf80 2023-10-30T12:45:14.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4815ad40 con 0x7f8f4815b910 2023-10-30T12:45:14.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481573b0 con 0x7f8f340039b0 2023-10-30T12:45:14.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34362/0 (socket says 172.21.15.8:34362) 2023-10-30T12:45:14.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 -- 172.21.15.8:0/3412270027 learned_addr learned my addr 172.21.15.8:0/3412270027 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:14.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 0x7f8f481ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f477fe640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4815b910 0x7f8f480c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 -- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 msgr2=0x7f8f481ef040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 0x7f8f481ef040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 -- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4815b910 msgr2=0x7f8f480c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4815b910 0x7f8f480c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 -- 172.21.15.8:0/3412270027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4815bd90 con 0x7f8f4815bf80 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 0x7f8f481ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3ffff640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f300072f0 tx=0x7f8f300077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30007c70 con 0x7f8f4815bf80 2023-10-30T12:45:14.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3000f8e0 con 0x7f8f4815bf80 2023-10-30T12:45:14.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4815c4d0 con 0x7f8f4815bf80 2023-10-30T12:45:14.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3000fc70 con 0x7f8f4815bf80 2023-10-30T12:45:14.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48156d60 con 0x7f8f4815bf80 2023-10-30T12:45:14.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.510+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f48159b90 con 0x7f8f4815bf80 2023-10-30T12:45:14.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.514+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8f3001e3b0 con 0x7f8f4815bf80 2023-10-30T12:45:14.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.514+0000 7f8f3f7fe640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 0x7f8f28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.514+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f30083690 con 0x7f8f4815bf80 2023-10-30T12:45:14.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.514+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 0x7f8f28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.514+0000 7f8f46ffd640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 0x7f8f28068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f380ad000 tx=0x7f8f380af040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.518+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8f48159b90 con 0x7f8f4815bf80 2023-10-30T12:45:14.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.706+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f480c7570 con 0x7f8f4815bf80 2023-10-30T12:45:14.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f3f7fe640 1 -- 172.21.15.8:0/3412270027 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5262 (secure 0 0 0) 0x7f8f480c7570 con 0x7f8f4815bf80 2023-10-30T12:45:14.714 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:14.715 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4303,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-30T12:45:14.716 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 3 2023-10-30T12:45:14.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 msgr2=0x7f8f28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 0x7f8f28068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f380ad000 tx=0x7f8f380af040 comp rx=0 tx=0).stop 2023-10-30T12:45:14.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 msgr2=0x7f8f4815b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f300072f0 tx=0x7f8f300077b0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 shutdown_connections 2023-10-30T12:45:14.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f280664a0 0x7f8f28068960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f4815b910 0x7f8f480c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f4815bf80 0x7f8f4815b3d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 --2- 172.21.15.8:0/3412270027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f340039b0 0x7f8f481ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 >> 172.21.15.8:0/3412270027 conn(0x7f8f480715c0 msgr2=0x7f8f48152000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:14.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 shutdown_connections 2023-10-30T12:45:14.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.710+0000 7f8f4d69c640 1 -- 172.21.15.8:0/3412270027 wait complete. 2023-10-30T12:45:14.735 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:14.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 Processor -- start 2023-10-30T12:45:14.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 -- start start 2023-10-30T12:45:14.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb1341597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13415a770 con 0x7fb1340cb180 2023-10-30T12:45:14.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340c9d40 con 0x7fb1340c62b0 2023-10-30T12:45:14.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb13a303640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134159cf0 con 0x7fb134153b50 2023-10-30T12:45:14.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fb1340cb180 0x7fb13406ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43626/0 (socket says 172.21.15.8:43626) 2023-10-30T12:45:14.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 -- 172.21.15.8:0/3329475691 learned_addr learned my addr 172.21.15.8:0/3329475691 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:14.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139301640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb1341597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374984344 0 0) 0x7fb12c0014a0 con 0x7fb1340cb180 2023-10-30T12:45:14.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb120003530 con 0x7fb1340cb180 2023-10-30T12:45:14.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778261827 0 0) 0x7fb1340c9d40 con 0x7fb1340c62b0 2023-10-30T12:45:14.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12c0014a0 con 0x7fb1340c62b0 2023-10-30T12:45:14.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895053900 0 0) 0x7fb120003530 con 0x7fb1340cb180 2023-10-30T12:45:14.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 msgr2=0x7fb1341597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb1341597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 >> v1:172.21.15.46:6789/0 conn(0x7fb1340c62b0 legacy=0x7fb1340ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134153f30 con 0x7fb1340cb180 2023-10-30T12:45:14.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fb12c001b90 con 0x7fb1340cb180 2023-10-30T12:45:14.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 >> v1:172.21.15.8:6789/0 conn(0x7fb1340cb180 legacy=0x7fb13406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb120003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1200040f0 0x7fb1200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb120004d10 0x7fb120005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134159cf0 con 0x7fb120004d10 2023-10-30T12:45:14.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12000bad0 con 0x7fb1200040f0 2023-10-30T12:45:14.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12000bc90 con 0x7fb134153b50 2023-10-30T12:45:14.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb120004d10 0x7fb120005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139301640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb120003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb138b00640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1200040f0 0x7fb1200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 -- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 msgr2=0x7fb120003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb134153b50 0x7fb120003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 -- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1200040f0 msgr2=0x7fb1200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:14.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1200040f0 0x7fb1200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.874+0000 7fb139b02640 1 -- 172.21.15.8:0/3329475691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120003530 con 0x7fb120004d10 2023-10-30T12:45:14.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb139b02640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb120004d10 0x7fb120005190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00a710 tx=0x7fb12c01f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29404bad5350149e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0072d0 con 0x7fb120004d10 2023-10-30T12:45:14.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c007430 con 0x7fb120004d10 2023-10-30T12:45:14.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb12bfff640 1 -- 172.21.15.8:0/3329475691 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c007860 con 0x7fb120004d10 2023-10-30T12:45:14.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 -- 172.21.15.8:0/3329475691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb120004d10 msgr2=0x7fb120005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 --2- 172.21.15.8:0/3329475691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb120004d10 0x7fb120005190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00a710 tx=0x7fb12c01f940 comp rx=0 tx=0).stop 2023-10-30T12:45:14.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb139b02640 1 -- 172.21.15.8:0/3329475691 reap_dead start 2023-10-30T12:45:14.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 -- 172.21.15.8:0/3329475691 shutdown_connections 2023-10-30T12:45:14.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 -- 172.21.15.8:0/3329475691 >> 172.21.15.8:0/3329475691 conn(0x7fb134065a80 msgr2=0x7fb134065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 -- 172.21.15.8:0/3329475691 shutdown_connections 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.878+0000 7fb13a303640 1 -- 172.21.15.8:0/3329475691 wait complete. 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 Processor -- start 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- start start 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb120004d10 0x7fb1341553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb1341eebd0 0x7fb1341e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134154120 con 0x7fb1341e63b0 2023-10-30T12:45:14.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13414d270 con 0x7fb120004d10 2023-10-30T12:45:14.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134154330 con 0x7fb1341eebd0 2023-10-30T12:45:14.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34392/0 (socket says 172.21.15.8:34392) 2023-10-30T12:45:14.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 -- 172.21.15.8:0/2515394896 learned_addr learned my addr 172.21.15.8:0/2515394896 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:14.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb138b00640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb1341eebd0 0x7fb1341e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139301640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb120004d10 0x7fb1341553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 -- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb1341eebd0 msgr2=0x7fb1341e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb1341eebd0 0x7fb1341e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 -- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb120004d10 msgr2=0x7fb1341553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:14.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb120004d10 0x7fb1341553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:14.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 -- 172.21.15.8:0/2515394896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134154610 con 0x7fb1341e63b0 2023-10-30T12:45:14.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb139b02640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00a0d0 tx=0x7fb120005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0072d0 con 0x7fb1341e63b0 2023-10-30T12:45:14.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c0ba070 con 0x7fb1341e63b0 2023-10-30T12:45:14.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1341548f0 con 0x7fb1341e63b0 2023-10-30T12:45:14.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004d70 con 0x7fb1341e63b0 2023-10-30T12:45:14.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.882+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb134154eb0 con 0x7fb1341e63b0 2023-10-30T12:45:14.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb134067540 con 0x7fb1341e63b0 2023-10-30T12:45:14.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb12c0075a0 con 0x7fb1341e63b0 2023-10-30T12:45:14.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb129ffb640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 0x7fb118068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:14.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb12c0bc070 con 0x7fb1341e63b0 2023-10-30T12:45:14.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb139301640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 0x7fb118068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:14.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.886+0000 7fb139301640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 0x7fb118068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb130009be0 tx=0x7fb130009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:14.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:14.890+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb12c0ef280 con 0x7fb1341e63b0 2023-10-30T12:45:15.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.054+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb13415a770 con 0x7fb1341e63b0 2023-10-30T12:45:15.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.054+0000 7fb129ffb640 1 -- 172.21.15.8:0/2515394896 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12730 (secure 0 0 0) 0x7fb13415a770 con 0x7fb1341e63b0 2023-10-30T12:45:15.060 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:15.060 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":16,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:08.010934+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:15.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 msgr2=0x7fb118068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 0x7fb118068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb130009be0 tx=0x7fb130009190 comp rx=0 tx=0).stop 2023-10-30T12:45:15.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 msgr2=0x7fb1340c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00a0d0 tx=0x7fb120005800 comp rx=0 tx=0).stop 2023-10-30T12:45:15.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 shutdown_connections 2023-10-30T12:45:15.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb1180664a0 0x7fb118068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1341e63b0 0x7fb1340c4740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb1341eebd0 0x7fb1341e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 --2- 172.21.15.8:0/2515394896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb120004d10 0x7fb1341553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 >> 172.21.15.8:0/2515394896 conn(0x7fb134065a80 msgr2=0x7fb13414e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 shutdown_connections 2023-10-30T12:45:15.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.058+0000 7fb13a303640 1 -- 172.21.15.8:0/2515394896 wait complete. 2023-10-30T12:45:15.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:15.082 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:15.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 Processor -- start 2023-10-30T12:45:15.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 -- start start 2023-10-30T12:45:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0314153ab0 0x7f03140b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f03141598b0 0x7f03140c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03140b2700 con 0x7f0314159cb0 2023-10-30T12:45:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140b21c0 con 0x7f03141598b0 2023-10-30T12:45:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031415a140 con 0x7f0314153ab0 2023-10-30T12:45:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f03141598b0 0x7f03140c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f03141598b0 0x7f03140c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34408/0 (socket says 172.21.15.8:34408) 2023-10-30T12:45:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 -- 172.21.15.8:0/1798664564 learned_addr learned my addr 172.21.15.8:0/1798664564 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f03137fe640 1 --1- 172.21.15.8:0/1798664564 >> v1:172.21.15.8:6789/0 conn(0x7f0314159cb0 0x7f03140b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43642/0 (socket says 172.21.15.8:43642) 2023-10-30T12:45:15.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f0312ffd640 1 --2- 172.21.15.8:0/1798664564 >> v2:172.21.15.46:3300/0 conn(0x7f0314153ab0 0x7f03140b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 -- 172.21.15.8:0/1798664564 >> v2:172.21.15.46:3300/0 conn(0x7f0314153ab0 msgr2=0x7f03140b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:15.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 --2- 172.21.15.8:0/1798664564 >> v2:172.21.15.46:3300/0 conn(0x7f0314153ab0 0x7f03140b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 -- 172.21.15.8:0/1798664564 >> v1:172.21.15.8:6789/0 conn(0x7f0314159cb0 legacy=0x7f03140b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 -- 172.21.15.8:0/1798664564 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031415a420 con 0x7f03141598b0 2023-10-30T12:45:15.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f03127fc640 1 -- 172.21.15.8:0/1798664564 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445392925 0 0) 0x7f03140b2700 con 0x7f0314159cb0 2023-10-30T12:45:15.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f030bfff640 1 --2- 172.21.15.8:0/1798664564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f03141598b0 0x7f03140c9e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03140b51b0 tx=0x7f02fc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80deb65b37ebc01 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f03127fc640 1 -- 172.21.15.8:0/1798664564 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0a89f0 con 0x7f03141598b0 2023-10-30T12:45:15.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f03127fc640 1 -- 172.21.15.8:0/1798664564 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0a8b50 con 0x7f03141598b0 2023-10-30T12:45:15.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f03127fc640 1 -- 172.21.15.8:0/1798664564 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b0440 con 0x7f03141598b0 2023-10-30T12:45:15.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 -- 172.21.15.8:0/1798664564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f03141598b0 msgr2=0x7f03140c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.226+0000 7f031986b640 1 --2- 172.21.15.8:0/1798664564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f03141598b0 0x7f03140c9e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03140b51b0 tx=0x7f02fc01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- 172.21.15.8:0/1798664564 shutdown_connections 2023-10-30T12:45:15.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 --2- 172.21.15.8:0/1798664564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f03141598b0 0x7f03140c9e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 --2- 172.21.15.8:0/1798664564 >> v2:172.21.15.46:3300/0 conn(0x7f0314153ab0 0x7f03140b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- 172.21.15.8:0/1798664564 >> 172.21.15.8:0/1798664564 conn(0x7f031406bef0 msgr2=0x7f031406c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- 172.21.15.8:0/1798664564 shutdown_connections 2023-10-30T12:45:15.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- 172.21.15.8:0/1798664564 wait complete. 2023-10-30T12:45:15.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 Processor -- start 2023-10-30T12:45:15.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- start start 2023-10-30T12:45:15.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 0x7f03141f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0314150440 0x7f03141f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031414d8b0 con 0x7f0314153ab0 2023-10-30T12:45:15.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031406c4e0 con 0x7f03141598b0 2023-10-30T12:45:15.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03141f2b80 con 0x7f0314150440 2023-10-30T12:45:15.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34418/0 (socket says 172.21.15.8:34418) 2023-10-30T12:45:15.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 -- 172.21.15.8:0/2293360348 learned_addr learned my addr 172.21.15.8:0/2293360348 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:15.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f030bfff640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 0x7f03141f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f03137fe640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0314150440 0x7f03141f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 -- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0314150440 msgr2=0x7f03141f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0314150440 0x7f03141f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 -- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 msgr2=0x7f03141f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 0x7f03141f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 -- 172.21.15.8:0/2293360348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0094d0 con 0x7f0314153ab0 2023-10-30T12:45:15.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f030bfff640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 0x7f03141f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:15.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0312ffd640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f030400d800 tx=0x7f030400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030400b6c0 con 0x7f0314153ab0 2023-10-30T12:45:15.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030400b820 con 0x7f0314153ab0 2023-10-30T12:45:15.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030400bb90 con 0x7f0314153ab0 2023-10-30T12:45:15.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.230+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03141f2e60 con 0x7f0314153ab0 2023-10-30T12:45:15.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031414d5d0 con 0x7f0314153ab0 2023-10-30T12:45:15.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031404e170 con 0x7f0314153ab0 2023-10-30T12:45:15.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f030400bcf0 con 0x7f0314153ab0 2023-10-30T12:45:15.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f0310ff9640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 0x7f02f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0304084260 con 0x7f0314153ab0 2023-10-30T12:45:15.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f030bfff640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 0x7f02f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.234+0000 7f030bfff640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 0x7f02f4068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0b2000 tx=0x7f02fc0aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.238+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0304051880 con 0x7f0314153ab0 2023-10-30T12:45:15.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03140caa00 con 0x7f0314153ab0 2023-10-30T12:45:15.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f0310ff9640 1 -- 172.21.15.8:0/2293360348 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0304082e60 con 0x7f0314153ab0 2023-10-30T12:45:15.403 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:15.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 msgr2=0x7f02f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 0x7f02f4068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0b2000 tx=0x7f02fc0aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:15.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 msgr2=0x7f03141f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f030400d800 tx=0x7f030400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 shutdown_connections 2023-10-30T12:45:15.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02f40664a0 0x7f02f4068960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03141598b0 0x7f03141f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0314150440 0x7f03141f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 --2- 172.21.15.8:0/2293360348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0314153ab0 0x7f03141f22b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 >> 172.21.15.8:0/2293360348 conn(0x7f031406bef0 msgr2=0x7f03140cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 shutdown_connections 2023-10-30T12:45:15.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.398+0000 7f031986b640 1 -- 172.21.15.8:0/2293360348 wait complete. 2023-10-30T12:45:15.424 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-30T12:45:15.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 Processor -- start 2023-10-30T12:45:15.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 -- start start 2023-10-30T12:45:15.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f40153c80 0x7f6f4015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6f400c4a10 0x7f6f400c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f4015a3b0 0x7f6f400ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4015dec0 con 0x7f6f400c4a10 2023-10-30T12:45:15.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f401599a0 con 0x7f6f40153c80 2023-10-30T12:45:15.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f47e17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40159f20 con 0x7f6f4015a3b0 2023-10-30T12:45:15.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6f400c4a10 0x7f6f400c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6f400c4a10 0x7f6f400c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34426/0 (socket says 172.21.15.8:34426) 2023-10-30T12:45:15.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 -- 172.21.15.8:0/2397472715 learned_addr learned my addr 172.21.15.8:0/2397472715 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:15.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f45bb3640 1 --2- 172.21.15.8:0/2397472715 >> v2:172.21.15.46:3300/0 conn(0x7f6f40153c80 0x7f6f4015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f463b4640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f4015a3b0 0x7f6f400ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 -- 172.21.15.8:0/2397472715 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f4015a3b0 msgr2=0x7f6f400ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:15.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f4015a3b0 0x7f6f400ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 -- 172.21.15.8:0/2397472715 >> v2:172.21.15.46:3300/0 conn(0x7f6f40153c80 msgr2=0x7f6f4015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:15.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 --2- 172.21.15.8:0/2397472715 >> v2:172.21.15.46:3300/0 conn(0x7f6f40153c80 0x7f6f4015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.566+0000 7f6f453b2640 1 -- 172.21.15.8:0/2397472715 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400c47e0 con 0x7f6f400c4a10 2023-10-30T12:45:15.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f453b2640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f400c4a10 0x7f6f400c42a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f400d00f0 tx=0x7f6f3001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a200c5c5396f2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f44bb1640 1 -- 172.21.15.8:0/2397472715 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300a89b0 con 0x7f6f400c4a10 2023-10-30T12:45:15.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f44bb1640 1 -- 172.21.15.8:0/2397472715 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f300a8b10 con 0x7f6f400c4a10 2023-10-30T12:45:15.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f44bb1640 1 -- 172.21.15.8:0/2397472715 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300a8ea0 con 0x7f6f400c4a10 2023-10-30T12:45:15.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 -- 172.21.15.8:0/2397472715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f400c4a10 msgr2=0x7f6f400c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f400c4a10 0x7f6f400c42a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f400d00f0 tx=0x7f6f3001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 -- 172.21.15.8:0/2397472715 shutdown_connections 2023-10-30T12:45:15.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f4015a3b0 0x7f6f400ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 --2- 172.21.15.8:0/2397472715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f400c4a10 0x7f6f400c42a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 --2- 172.21.15.8:0/2397472715 >> v2:172.21.15.46:3300/0 conn(0x7f6f40153c80 0x7f6f4015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 -- 172.21.15.8:0/2397472715 >> 172.21.15.8:0/2397472715 conn(0x7f6f400715c0 msgr2=0x7f6f400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 -- 172.21.15.8:0/2397472715 shutdown_connections 2023-10-30T12:45:15.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.570+0000 7f6f47e17640 1 -- 172.21.15.8:0/2397472715 wait complete. 2023-10-30T12:45:15.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 Processor -- start 2023-10-30T12:45:15.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- start start 2023-10-30T12:45:15.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f400c4a10 0x7f6f400ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015a3b0 0x7f6f400b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4015c410 con 0x7f6f40153c80 2023-10-30T12:45:15.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400b2600 con 0x7f6f4015a3b0 2023-10-30T12:45:15.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400c5d90 con 0x7f6f400c4a10 2023-10-30T12:45:15.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34442/0 (socket says 172.21.15.8:34442) 2023-10-30T12:45:15.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 -- 172.21.15.8:0/3779926476 learned_addr learned my addr 172.21.15.8:0/3779926476 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:15.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f45bb3640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f400c4a10 0x7f6f400ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f463b4640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015a3b0 0x7f6f400b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 -- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f400c4a10 msgr2=0x7f6f400ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f400c4a10 0x7f6f400ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 -- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015a3b0 msgr2=0x7f6f400b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015a3b0 0x7f6f400b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 -- 172.21.15.8:0/3779926476 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300094d0 con 0x7f6f40153c80 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f453b2640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f3001d3a0 tx=0x7f6f3001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300ba600 con 0x7f6f40153c80 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f300ba760 con 0x7f6f40153c80 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4015b840 con 0x7f6f40153c80 2023-10-30T12:45:15.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300bab10 con 0x7f6f40153c80 2023-10-30T12:45:15.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.574+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4015bdc0 con 0x7f6f40153c80 2023-10-30T12:45:15.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4015c580 con 0x7f6f40153c80 2023-10-30T12:45:15.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f300bac70 con 0x7f6f40153c80 2023-10-30T12:45:15.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f367fc640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 0x7f6f24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f30122250 con 0x7f6f40153c80 2023-10-30T12:45:15.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f45bb3640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 0x7f6f24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.578+0000 7f6f45bb3640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 0x7f6f24068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00ba30 tx=0x7f6f3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.582+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6f300efa10 con 0x7f6f40153c80 2023-10-30T12:45:15.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6f4004e170 con 0x7f6f40153c80 2023-10-30T12:45:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f367fc640 1 -- 172.21.15.8:0/3779926476 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6f300f3030 con 0x7f6f40153c80 2023-10-30T12:45:15.751 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:15.752 INFO:teuthology.orchestra.run.smithi008.stdout:[ 2023-10-30T12:45:15.752 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-10-30T12:45:15.752 INFO:teuthology.orchestra.run.smithi008.stdout: "poolnum": 1, 2023-10-30T12:45:15.752 INFO:teuthology.orchestra.run.smithi008.stdout: "poolname": ".mgr" 2023-10-30T12:45:15.752 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-10-30T12:45:15.753 INFO:teuthology.orchestra.run.smithi008.stdout:] 2023-10-30T12:45:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 msgr2=0x7f6f24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 0x7f6f24068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00ba30 tx=0x7f6f3c008040 comp rx=0 tx=0).stop 2023-10-30T12:45:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 msgr2=0x7f6f400ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f3001d3a0 tx=0x7f6f3001ba90 comp rx=0 tx=0).stop 2023-10-30T12:45:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 shutdown_connections 2023-10-30T12:45:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6f240664a0 0x7f6f24068960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f4015a3b0 0x7f6f400b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6f40153c80 0x7f6f400ceae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 --2- 172.21.15.8:0/3779926476 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6f400c4a10 0x7f6f400ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 >> 172.21.15.8:0/3779926476 conn(0x7f6f400715c0 msgr2=0x7f6f400719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 shutdown_connections 2023-10-30T12:45:15.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.750+0000 7f6f47e17640 1 -- 172.21.15.8:0/3779926476 wait complete. 2023-10-30T12:45:15.772 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-30T12:45:15.772 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-30T12:45:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 Processor -- start 2023-10-30T12:45:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 -- start start 2023-10-30T12:45:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f101c1599e0 0x7f101c153880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c06af00 con 0x7f101c0c4720 2023-10-30T12:45:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c06a660 con 0x7f101c0c4340 2023-10-30T12:45:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f102230f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c0c4cd0 con 0x7f101c1599e0 2023-10-30T12:45:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f10208ac640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f101c1599e0 0x7f101c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f101b7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f101c0c4720 0x7f101c0ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43654/0 (socket says 172.21.15.8:43654) 2023-10-30T12:45:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f101b7fe640 1 -- 172.21.15.8:0/1492459031 learned_addr learned my addr 172.21.15.8:0/1492459031 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f10208ac640 1 -- 172.21.15.8:0/1492459031 >> v1:172.21.15.46:6789/0 conn(0x7f101c0c4340 legacy=0x7f101c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f10208ac640 1 -- 172.21.15.8:0/1492459031 >> v1:172.21.15.8:6789/0 conn(0x7f101c0c4720 legacy=0x7f101c0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f10208ac640 1 -- 172.21.15.8:0/1492459031 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c06a940 con 0x7f101c1599e0 2023-10-30T12:45:15.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.918+0000 7f101affd640 1 -- 172.21.15.8:0/1492459031 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143542718 0 0) 0x7f101c06af00 con 0x7f101c0c4720 2023-10-30T12:45:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f10208ac640 1 --2- 172.21.15.8:0/1492459031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c1599e0 0x7f101c153880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1014009d90 tx=0x7f101401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb8d66b8d2a6115 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f101affd640 1 -- 172.21.15.8:0/1492459031 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140a5860 con 0x7f101c1599e0 2023-10-30T12:45:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f101affd640 1 -- 172.21.15.8:0/1492459031 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10140a59c0 con 0x7f101c1599e0 2023-10-30T12:45:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f101affd640 1 -- 172.21.15.8:0/1492459031 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140a5d50 con 0x7f101c1599e0 2023-10-30T12:45:15.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 -- 172.21.15.8:0/1492459031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c1599e0 msgr2=0x7f101c153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 --2- 172.21.15.8:0/1492459031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c1599e0 0x7f101c153880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1014009d90 tx=0x7f101401c890 comp rx=0 tx=0).stop 2023-10-30T12:45:15.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 -- 172.21.15.8:0/1492459031 shutdown_connections 2023-10-30T12:45:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 --2- 172.21.15.8:0/1492459031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c1599e0 0x7f101c153880 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 -- 172.21.15.8:0/1492459031 >> 172.21.15.8:0/1492459031 conn(0x7f101c0cf1c0 msgr2=0x7f101c0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 -- 172.21.15.8:0/1492459031 shutdown_connections 2023-10-30T12:45:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.922+0000 7f102230f640 1 -- 172.21.15.8:0/1492459031 wait complete. 2023-10-30T12:45:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 Processor -- start 2023-10-30T12:45:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- start start 2023-10-30T12:45:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 0x7f101c142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 0x7f101c146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c147f00 con 0x7f101c0c4340 2023-10-30T12:45:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1f5240 con 0x7f101c0c4720 2023-10-30T12:45:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c14e6f0 con 0x7f101c1599e0 2023-10-30T12:45:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34460/0 (socket says 172.21.15.8:34460) 2023-10-30T12:45:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 -- 172.21.15.8:0/1844243271 learned_addr learned my addr 172.21.15.8:0/1844243271 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101b7fe640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 0x7f101c142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f10208ac640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 0x7f101c146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 -- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 msgr2=0x7f101c146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 0x7f101c146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 -- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 msgr2=0x7f101c142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 0x7f101c142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 -- 172.21.15.8:0/1844243271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014009a70 con 0x7f101c0c4340 2023-10-30T12:45:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f10208ac640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 0x7f101c146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101b7fe640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 0x7f101c142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:15.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f101bfff640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c3c0 tx=0x7f100c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0052d0 con 0x7f101c0c4340 2023-10-30T12:45:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c005430 con 0x7f101c0c4340 2023-10-30T12:45:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c14e990 con 0x7f101c0c4340 2023-10-30T12:45:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c003840 con 0x7f101c0c4340 2023-10-30T12:45:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c0ceb30 con 0x7f101c0c4340 2023-10-30T12:45:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.926+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c04e170 con 0x7f101c0c4340 2023-10-30T12:45:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.930+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f100c0039a0 con 0x7f101c0c4340 2023-10-30T12:45:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.930+0000 7f1018ff9640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 0x7f1008068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.930+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f100c083180 con 0x7f101c0c4340 2023-10-30T12:45:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.930+0000 7f101b7fe640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 0x7f1008068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.930+0000 7f101b7fe640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 0x7f1008068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f101c0c8b30 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:15.934+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f100c050820 con 0x7f101c0c4340 2023-10-30T12:45:16.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:16.094+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f101c0cb660 con 0x7f101c0c4340 2023-10-30T12:45:17.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.045+0000 7f1018ff9640 1 -- 172.21.15.8:0/1844243271 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f100c053e40 con 0x7f101c0c4340 2023-10-30T12:45:17.050 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' created 2023-10-30T12:45:17.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 msgr2=0x7f1008068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 0x7f1008068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f101c0c8b30 tx=0x7f1010009040 comp rx=0 tx=0).stop 2023-10-30T12:45:17.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 msgr2=0x7f101c148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c3c0 tx=0x7f100c00c880 comp rx=0 tx=0).stop 2023-10-30T12:45:17.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 shutdown_connections 2023-10-30T12:45:17.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1008066470 0x7f1008068930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f101c0c4720 0x7f101c142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f101c1599e0 0x7f101c146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 --2- 172.21.15.8:0/1844243271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f101c0c4340 0x7f101c148d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 >> 172.21.15.8:0/1844243271 conn(0x7f101c0cf1c0 msgr2=0x7f101c0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 shutdown_connections 2023-10-30T12:45:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.049+0000 7f102230f640 1 -- 172.21.15.8:0/1844243271 wait complete. 2023-10-30T12:45:17.082 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-30T12:45:17.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.233+0000 7f0f4b017640 1 Processor -- start 2023-10-30T12:45:17.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.233+0000 7f0f4b017640 1 -- start start 2023-10-30T12:45:17.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4b017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f440c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4b017640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f440cb380 con 0x7f0f4415d520 2023-10-30T12:45:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4b017640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4415dd10 con 0x7f0f440ca550 2023-10-30T12:45:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4b017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415da30 con 0x7f0f440ca170 2023-10-30T12:45:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4a816640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0f4415d520 0x7f0f44153590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43656/0 (socket says 172.21.15.8:43656) 2023-10-30T12:45:17.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4a816640 1 -- 172.21.15.8:0/1075457062 learned_addr learned my addr 172.21.15.8:0/1075457062 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4a015640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f440c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362691432 0 0) 0x7f0f440cb380 con 0x7f0f4415d520 2023-10-30T12:45:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f30003530 con 0x7f0f4415d520 2023-10-30T12:45:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299354680 0 0) 0x7f0f4415dd10 con 0x7f0f440ca550 2023-10-30T12:45:17.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f440cb380 con 0x7f0f440ca550 2023-10-30T12:45:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248780172 0 0) 0x7f0f30003530 con 0x7f0f4415d520 2023-10-30T12:45:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 msgr2=0x7f0f440c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f440c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 >> v1:172.21.15.46:6789/0 conn(0x7f0f440ca550 legacy=0x7f0f4415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440c4810 con 0x7f0f4415d520 2023-10-30T12:45:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f0f3c001a00 con 0x7f0f4415d520 2023-10-30T12:45:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 >> v1:172.21.15.8:6789/0 conn(0x7f0f4415d520 legacy=0x7f0f44153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f30003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f300040f0 0x7f0f300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f30004d10 0x7f0f30005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415da30 con 0x7f0f300040f0 2023-10-30T12:45:17.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3000bad0 con 0x7f0f30004d10 2023-10-30T12:45:17.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3000bc90 con 0x7f0f440ca170 2023-10-30T12:45:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f300040f0 0x7f0f300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4a015640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f30003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f4a816640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f30004d10 0x7f0f30005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 -- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 msgr2=0x7f0f30003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:17.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f440ca170 0x7f0f30003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 -- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f30004d10 msgr2=0x7f0f30005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f30004d10 0x7f0f30005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.237+0000 7f0f49814640 1 -- 172.21.15.8:0/1075457062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30003530 con 0x7f0f300040f0 2023-10-30T12:45:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f49814640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f300040f0 0x7f0f300047d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001930 tx=0x7f0f3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df43cc78bb3e7b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:17.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340052d0 con 0x7f0f300040f0 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34005430 con 0x7f0f300040f0 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f49013640 1 -- 172.21.15.8:0/1075457062 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34003870 con 0x7f0f300040f0 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 -- 172.21.15.8:0/1075457062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f300040f0 msgr2=0x7f0f300047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 --2- 172.21.15.8:0/1075457062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f300040f0 0x7f0f300047d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001930 tx=0x7f0f3401c4a0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4a816640 1 -- 172.21.15.8:0/1075457062 reap_dead start 2023-10-30T12:45:17.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 -- 172.21.15.8:0/1075457062 shutdown_connections 2023-10-30T12:45:17.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 -- 172.21.15.8:0/1075457062 >> 172.21.15.8:0/1075457062 conn(0x7f0f440715c0 msgr2=0x7f0f440719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:17.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 -- 172.21.15.8:0/1075457062 shutdown_connections 2023-10-30T12:45:17.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.241+0000 7f0f4b017640 1 -- 172.21.15.8:0/1075457062 wait complete. 2023-10-30T12:45:17.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 Processor -- start 2023-10-30T12:45:17.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- start start 2023-10-30T12:45:17.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f300040f0 0x7f0f4415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 0x7f0f440c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441efff0 con 0x7f0f440c7310 2023-10-30T12:45:17.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44150460 con 0x7f0f4415c3f0 2023-10-30T12:45:17.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415ba20 con 0x7f0f300040f0 2023-10-30T12:45:17.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34480/0 (socket says 172.21.15.8:34480) 2023-10-30T12:45:17.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 -- 172.21.15.8:0/2804188225 learned_addr learned my addr 172.21.15.8:0/2804188225 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:17.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4a015640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f300040f0 0x7f0f4415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4a816640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 0x7f0f440c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 -- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f300040f0 msgr2=0x7f0f4415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f300040f0 0x7f0f4415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 -- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 msgr2=0x7f0f440c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:17.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 0x7f0f440c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:17.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 -- 172.21.15.8:0/2804188225 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4415b3f0 con 0x7f0f440c7310 2023-10-30T12:45:17.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4a816640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 0x7f0f440c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:17.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f49814640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f340ae040 tx=0x7f0f30005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:17.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340bd070 con 0x7f0f440c7310 2023-10-30T12:45:17.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34007040 con 0x7f0f440c7310 2023-10-30T12:45:17.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340073d0 con 0x7f0f440c7310 2023-10-30T12:45:17.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f441506b0 con 0x7f0f440c7310 2023-10-30T12:45:17.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.245+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4415b6b0 con 0x7f0f440c7310 2023-10-30T12:45:17.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4415bb90 con 0x7f0f440c7310 2023-10-30T12:45:17.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f34007530 con 0x7f0f440c7310 2023-10-30T12:45:17.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f3affd640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 0x7f0f28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:17.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0f3411e5e0 con 0x7f0f440c7310 2023-10-30T12:45:17.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f4a015640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 0x7f0f28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:17.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.249+0000 7f0f4a015640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 0x7f0f28068960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009b00 tx=0x7f0f40009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:17.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.253+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f340ebcc0 con 0x7f0f440c7310 2023-10-30T12:45:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:17.413+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0f44067540 con 0x7f0f440c7310 2023-10-30T12:45:18.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.057+0000 7f0f3affd640 1 -- 172.21.15.8:0/2804188225 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0f340b53c0 con 0x7f0f440c7310 2023-10-30T12:45:18.060 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' created 2023-10-30T12:45:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 msgr2=0x7f0f28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 0x7f0f28068960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009b00 tx=0x7f0f40009190 comp rx=0 tx=0).stop 2023-10-30T12:45:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 msgr2=0x7f0f440c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f340ae040 tx=0x7f0f30005800 comp rx=0 tx=0).stop 2023-10-30T12:45:18.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 shutdown_connections 2023-10-30T12:45:18.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f280664a0 0x7f0f28068960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f4415c3f0 0x7f0f440c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f440c7310 0x7f0f440c7b10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 --2- 172.21.15.8:0/2804188225 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f300040f0 0x7f0f4415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 >> 172.21.15.8:0/2804188225 conn(0x7f0f440715c0 msgr2=0x7f0f4414e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:18.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 shutdown_connections 2023-10-30T12:45:18.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.061+0000 7f0f4b017640 1 -- 172.21.15.8:0/2804188225 wait complete. 2023-10-30T12:45:18.105 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-30T12:45:18.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 Processor -- start 2023-10-30T12:45:18.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 -- start start 2023-10-30T12:45:18.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa45c0c4810 0x7fa45c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45c153f80 con 0x7fa45c153ba0 2023-10-30T12:45:18.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c159df0 con 0x7fa45c0c4810 2023-10-30T12:45:18.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa4626d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c15a7c0 con 0x7fa45c0c4410 2023-10-30T12:45:18.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa45c0c4810 0x7fa45c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa45c0c4810 0x7fa45c15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34488/0 (socket says 172.21.15.8:34488) 2023-10-30T12:45:18.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 -- 172.21.15.8:0/66542927 learned_addr learned my addr 172.21.15.8:0/66542927 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:18.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa45b7fe640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 -- 172.21.15.8:0/66542927 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 msgr2=0x7fa45c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:18.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 -- 172.21.15.8:0/66542927 >> v1:172.21.15.46:6789/0 conn(0x7fa45c153ba0 legacy=0x7fa45c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.253+0000 7fa460c73640 1 -- 172.21.15.8:0/66542927 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c1597c0 con 0x7fa45c0c4810 2023-10-30T12:45:18.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa460c73640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c15a280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa454009e50 tx=0x7fa45401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b991c80b3d84c31 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:18.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa45affd640 1 -- 172.21.15.8:0/66542927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540a5810 con 0x7fa45c0c4810 2023-10-30T12:45:18.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa45affd640 1 -- 172.21.15.8:0/66542927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4540a5970 con 0x7fa45c0c4810 2023-10-30T12:45:18.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa45affd640 1 -- 172.21.15.8:0/66542927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4540a5d00 con 0x7fa45c0c4810 2023-10-30T12:45:18.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- 172.21.15.8:0/66542927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 msgr2=0x7fa45c15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c15a280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa454009e50 tx=0x7fa45401c950 comp rx=0 tx=0).stop 2023-10-30T12:45:18.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- 172.21.15.8:0/66542927 shutdown_connections 2023-10-30T12:45:18.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c15a280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 --2- 172.21.15.8:0/66542927 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- 172.21.15.8:0/66542927 >> 172.21.15.8:0/66542927 conn(0x7fa45c071790 msgr2=0x7fa45c071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:18.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- 172.21.15.8:0/66542927 shutdown_connections 2023-10-30T12:45:18.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- 172.21.15.8:0/66542927 wait complete. 2023-10-30T12:45:18.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 Processor -- start 2023-10-30T12:45:18.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.257+0000 7fa4626d6640 1 -- start start 2023-10-30T12:45:18.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 0x7fa45c143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c15c8a0 con 0x7fa45c0c4810 2023-10-30T12:45:18.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c1443b0 con 0x7fa45c153ba0 2023-10-30T12:45:18.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0c8e00 con 0x7fa45c0c4410 2023-10-30T12:45:18.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34496/0 (socket says 172.21.15.8:34496) 2023-10-30T12:45:18.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 -- 172.21.15.8:0/2088999915 learned_addr learned my addr 172.21.15.8:0/2088999915 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:18.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45bfff640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa460c73640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 0x7fa45c143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 -- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 msgr2=0x7fa45c153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 -- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 msgr2=0x7fa45c143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 0x7fa45c143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:18.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 -- 172.21.15.8:0/2088999915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009b30 con 0x7fa45c0c4810 2023-10-30T12:45:18.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa460c73640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 0x7fa45c143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:18.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa45b7fe640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e930 tx=0x7fa45000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:18.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450009690 con 0x7fa45c0c4810 2023-10-30T12:45:18.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45000ca00 con 0x7fa45c0c4810 2023-10-30T12:45:18.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45000cd70 con 0x7fa45c0c4810 2023-10-30T12:45:18.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c152290 con 0x7fa45c0c4810 2023-10-30T12:45:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1fa130 con 0x7fa45c0c4810 2023-10-30T12:45:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.261+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c04e170 con 0x7fa45c0c4810 2023-10-30T12:45:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.265+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa450013470 con 0x7fa45c0c4810 2023-10-30T12:45:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.265+0000 7fa458ff9640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 0x7fa448068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:18.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.265+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa450083f70 con 0x7fa45c0c4810 2023-10-30T12:45:18.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.265+0000 7fa45bfff640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 0x7fa448068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:18.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.265+0000 7fa45bfff640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 0x7fa448068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa44c007fd0 tx=0x7fa44c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:18.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.269+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4500515b0 con 0x7fa45c0c4810 2023-10-30T12:45:18.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:18.461+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa45c0713a0 con 0x7fa45c0c4810 2023-10-30T12:45:19.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa458ff9640 1 -- 172.21.15.8:0/2088999915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa4500170c0 con 0x7fa45c0c4810 2023-10-30T12:45:19.063 INFO:teuthology.orchestra.run.smithi008.stderr:new fs with metadata pool 2 and data pool 3 2023-10-30T12:45:19.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 msgr2=0x7fa448068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 0x7fa448068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa44c007fd0 tx=0x7fa44c009040 comp rx=0 tx=0).stop 2023-10-30T12:45:19.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 msgr2=0x7fa45c152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e930 tx=0x7fa45000edf0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 shutdown_connections 2023-10-30T12:45:19.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa4480664a0 0x7fa448068960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa45c153ba0 0x7fa45c143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa45c0c4810 0x7fa45c152ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 --2- 172.21.15.8:0/2088999915 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa45c0c4410 0x7fa45c153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.061+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 >> 172.21.15.8:0/2088999915 conn(0x7fa45c071790 msgr2=0x7fa45c158750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.065+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 shutdown_connections 2023-10-30T12:45:19.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.065+0000 7fa4626d6640 1 -- 172.21.15.8:0/2088999915 wait complete. 2023-10-30T12:45:19.113 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:19.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 Processor -- start 2023-10-30T12:45:19.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 -- start start 2023-10-30T12:45:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4fc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0cc900 con 0x7fe4fc153970 2023-10-30T12:45:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0cb040 con 0x7fe4fc15acd0 2023-10-30T12:45:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe50130f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cbfa0 con 0x7fe4fc153d50 2023-10-30T12:45:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4faffd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe4fc153970 0x7fe4fc15d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44680/0 (socket says 172.21.15.8:44680) 2023-10-30T12:45:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4faffd640 1 -- 172.21.15.8:0/2610328921 learned_addr learned my addr 172.21.15.8:0/2610328921 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4fc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248830724 0 0) 0x7fe4fc0cc900 con 0x7fe4fc153970 2023-10-30T12:45:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e4003530 con 0x7fe4fc153970 2023-10-30T12:45:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2541835737 0 0) 0x7fe4e4003530 con 0x7fe4fc153970 2023-10-30T12:45:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 msgr2=0x7fe4fc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4fc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 >> v1:172.21.15.46:6789/0 conn(0x7fe4fc15acd0 legacy=0x7fe4fc0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0cb320 con 0x7fe4fc153970 2023-10-30T12:45:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fe4ec001680 con 0x7fe4fc153970 2023-10-30T12:45:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 >> v1:172.21.15.8:6789/0 conn(0x7fe4fc153970 legacy=0x7fe4fc15d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4e4004d10 0x7fe4e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cbfa0 con 0x7fe4e4003f30 2023-10-30T12:45:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e400ba70 con 0x7fe4e4004d10 2023-10-30T12:45:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e400bc30 con 0x7fe4fc153d50 2023-10-30T12:45:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4faffd640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fb7fe640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4e4004d10 0x7fe4e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 -- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 msgr2=0x7fe4e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc153d50 0x7fe4e40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 -- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4e4004d10 msgr2=0x7fe4e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4e4004d10 0x7fe4e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.261+0000 7fe4fa7fc640 1 -- 172.21.15.8:0/2610328921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4003530 con 0x7fe4e4003f30 2023-10-30T12:45:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4e40047d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ae0 tx=0x7fe4f001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f706f9cdf9e5ec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00a5540 con 0x7fe4e4003f30 2023-10-30T12:45:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f00a56a0 con 0x7fe4e4003f30 2023-10-30T12:45:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe4f9ffb640 1 -- 172.21.15.8:0/2610328921 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00a5a50 con 0x7fe4e4003f30 2023-10-30T12:45:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 -- 172.21.15.8:0/2610328921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 msgr2=0x7fe4e40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 --2- 172.21.15.8:0/2610328921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4e40047d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ae0 tx=0x7fe4f001ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe4fb7fe640 1 -- 172.21.15.8:0/2610328921 reap_dead start 2023-10-30T12:45:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 -- 172.21.15.8:0/2610328921 shutdown_connections 2023-10-30T12:45:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 -- 172.21.15.8:0/2610328921 >> 172.21.15.8:0/2610328921 conn(0x7fe4fc06bf60 msgr2=0x7fe4fc06c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 -- 172.21.15.8:0/2610328921 shutdown_connections 2023-10-30T12:45:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.265+0000 7fe50130f640 1 -- 172.21.15.8:0/2610328921 wait complete. 2023-10-30T12:45:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 Processor -- start 2023-10-30T12:45:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- start start 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 0x7fe4fc1e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4fc1f1350 0x7fe4fc1f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc14c6e0 con 0x7fe4e4003f30 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f1d10 con 0x7fe4fc1f1350 2023-10-30T12:45:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc15a960 con 0x7fe4fc15a400 2023-10-30T12:45:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35926/0 (socket says 172.21.15.8:35926) 2023-10-30T12:45:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 -- 172.21.15.8:0/162263632 learned_addr learned my addr 172.21.15.8:0/162263632 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 0x7fe4fc1e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4fb7fe640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4fc1f1350 0x7fe4fc1f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 -- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 msgr2=0x7fe4fc1e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 0x7fe4fc1e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 -- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4fc1f1350 msgr2=0x7fe4fc1f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4fc1f1350 0x7fe4fc1f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 -- 172.21.15.8:0/162263632 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc1e2100 con 0x7fe4e4003f30 2023-10-30T12:45:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 0x7fe4fc1e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4faffd640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec001b50 tx=0x7fe4ec012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec010740 con 0x7fe4e4003f30 2023-10-30T12:45:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0108a0 con 0x7fe4e4003f30 2023-10-30T12:45:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc1e23e0 con 0x7fe4e4003f30 2023-10-30T12:45:19.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec010c50 con 0x7fe4e4003f30 2023-10-30T12:45:19.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.269+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc1e29a0 con 0x7fe4e4003f30 2023-10-30T12:45:19.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc067cb0 con 0x7fe4e4003f30 2023-10-30T12:45:19.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe4ec016480 con 0x7fe4e4003f30 2023-10-30T12:45:19.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe4e37fe640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 0x7fe4e8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4ec087380 con 0x7fe4e4003f30 2023-10-30T12:45:19.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 0x7fe4e8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.273+0000 7fe4fa7fc640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 0x7fe4e8068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00af000 tx=0x7fe4f00046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.277+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe4ec054760 con 0x7fe4e4003f30 2023-10-30T12:45:19.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.433+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4fc067540 con 0x7fe4e4003f30 2023-10-30T12:45:19.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe4e37fe640 1 -- 172.21.15.8:0/162263632 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16398 (secure 0 0 0) 0x7fe4ec057d80 con 0x7fe4e4003f30 2023-10-30T12:45:19.439 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:19.439 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":19,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:18.462708+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:19.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 msgr2=0x7fe4e8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 0x7fe4e8068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00af000 tx=0x7fe4f00046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 msgr2=0x7fe4fc14b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec001b50 tx=0x7fe4ec012c90 comp rx=0 tx=0).stop 2023-10-30T12:45:19.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 shutdown_connections 2023-10-30T12:45:19.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe4e80664d0 0x7fe4e8068990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4fc1f1350 0x7fe4fc1f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe4fc15a400 0x7fe4fc1e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 --2- 172.21.15.8:0/162263632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e4003f30 0x7fe4fc14b0b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 >> 172.21.15.8:0/162263632 conn(0x7fe4fc06bf60 msgr2=0x7fe4fc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 shutdown_connections 2023-10-30T12:45:19.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.437+0000 7fe50130f640 1 -- 172.21.15.8:0/162263632 wait complete. 2023-10-30T12:45:19.461 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:19.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.601+0000 7f600359e640 1 Processor -- start 2023-10-30T12:45:19.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.601+0000 7f600359e640 1 -- start start 2023-10-30T12:45:19.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600359e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f600415ac70 0x7f600415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600359e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f600406aeb0 0x7f600406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600359e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600415dcc0 con 0x7f60040c8eb0 2023-10-30T12:45:19.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600359e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6004159800 con 0x7f600406aeb0 2023-10-30T12:45:19.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600359e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6004159d80 con 0x7f600415ac70 2023-10-30T12:45:19.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f600406aeb0 0x7f600406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f600406aeb0 0x7f600406b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35930/0 (socket says 172.21.15.8:35930) 2023-10-30T12:45:19.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 -- 172.21.15.8:0/3172922243 learned_addr learned my addr 172.21.15.8:0/3172922243 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:19.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f600259c640 1 --1- 172.21.15.8:0/3172922243 >> v1:172.21.15.8:6789/0 conn(0x7f60040c8eb0 0x7f600415a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44692/0 (socket says 172.21.15.8:44692) 2023-10-30T12:45:19.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6001d9b640 1 --2- 172.21.15.8:0/3172922243 >> v2:172.21.15.46:3300/0 conn(0x7f600415ac70 0x7f600415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 -- 172.21.15.8:0/3172922243 >> v2:172.21.15.46:3300/0 conn(0x7f600415ac70 msgr2=0x7f600415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 --2- 172.21.15.8:0/3172922243 >> v2:172.21.15.46:3300/0 conn(0x7f600415ac70 0x7f600415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 -- 172.21.15.8:0/3172922243 >> v1:172.21.15.8:6789/0 conn(0x7f60040c8eb0 legacy=0x7f600415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.605+0000 7f6002d9d640 1 -- 172.21.15.8:0/3172922243 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600415d420 con 0x7f600406aeb0 2023-10-30T12:45:19.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f6002d9d640 1 --2- 172.21.15.8:0/3172922243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f600406b300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60040b4130 tx=0x7f5ffc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d487e83affe77 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600159a640 1 -- 172.21.15.8:0/3172922243 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0af400 con 0x7f600406aeb0 2023-10-30T12:45:19.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600159a640 1 -- 172.21.15.8:0/3172922243 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc0ab050 con 0x7f600406aeb0 2023-10-30T12:45:19.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600159a640 1 -- 172.21.15.8:0/3172922243 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0af780 con 0x7f600406aeb0 2023-10-30T12:45:19.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- 172.21.15.8:0/3172922243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 msgr2=0x7f600406b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 --2- 172.21.15.8:0/3172922243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f600406b300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60040b4130 tx=0x7f5ffc01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- 172.21.15.8:0/3172922243 shutdown_connections 2023-10-30T12:45:19.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 --2- 172.21.15.8:0/3172922243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f600406b300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 --2- 172.21.15.8:0/3172922243 >> v2:172.21.15.46:3300/0 conn(0x7f600415ac70 0x7f600415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- 172.21.15.8:0/3172922243 >> 172.21.15.8:0/3172922243 conn(0x7f60040cef00 msgr2=0x7f60040cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- 172.21.15.8:0/3172922243 shutdown_connections 2023-10-30T12:45:19.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- 172.21.15.8:0/3172922243 wait complete. 2023-10-30T12:45:19.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 Processor -- start 2023-10-30T12:45:19.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.609+0000 7f600359e640 1 -- start start 2023-10-30T12:45:19.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f60040c8eb0 0x7f600414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f600415ac70 0x7f600414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60041f2b90 con 0x7f600406aeb0 2023-10-30T12:45:19.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60041eeb90 con 0x7f600415ac70 2023-10-30T12:45:19.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60041f30c0 con 0x7f60040c8eb0 2023-10-30T12:45:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35944/0 (socket says 172.21.15.8:35944) 2023-10-30T12:45:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 -- 172.21.15.8:0/452242594 learned_addr learned my addr 172.21.15.8:0/452242594 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f6002d9d640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f600415ac70 0x7f600414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f6001d9b640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f60040c8eb0 0x7f600414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 -- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f60040c8eb0 msgr2=0x7f600414d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f60040c8eb0 0x7f600414d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 -- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f600415ac70 msgr2=0x7f600414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f600415ac70 0x7f600414d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 -- 172.21.15.8:0/452242594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc009b30 con 0x7f600406aeb0 2023-10-30T12:45:19.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600259c640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c490 tx=0x7f5ff400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4007480 con 0x7f600406aeb0 2023-10-30T12:45:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff40075e0 con 0x7f600406aeb0 2023-10-30T12:45:19.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6004155e30 con 0x7f600406aeb0 2023-10-30T12:45:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4004a10 con 0x7f600406aeb0 2023-10-30T12:45:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60041560d0 con 0x7f600406aeb0 2023-10-30T12:45:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600404e170 con 0x7f600406aeb0 2023-10-30T12:45:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.613+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ff4004b70 con 0x7f600406aeb0 2023-10-30T12:45:19.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.617+0000 7f5ff2ffd640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 0x7f5fe4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.617+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ff40830f0 con 0x7f600406aeb0 2023-10-30T12:45:19.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.617+0000 7f6001d9b640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 0x7f5fe4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.617+0000 7f6001d9b640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 0x7f5fe4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6004157300 tx=0x7f5ff8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.621+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ff40505e0 con 0x7f600406aeb0 2023-10-30T12:45:19.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f600415dcc0 con 0x7f600406aeb0 2023-10-30T12:45:19.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f5ff2ffd640 1 -- 172.21.15.8:0/452242594 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16398 (secure 0 0 0) 0x7f600415dcc0 con 0x7f600406aeb0 2023-10-30T12:45:19.784 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":19,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:18.462708+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:19.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 msgr2=0x7f5fe4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 0x7f5fe4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6004157300 tx=0x7f5ff8008040 comp rx=0 tx=0).stop 2023-10-30T12:45:19.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 msgr2=0x7f60040c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c490 tx=0x7f5ff400c950 comp rx=0 tx=0).stop 2023-10-30T12:45:19.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 shutdown_connections 2023-10-30T12:45:19.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5fe40664d0 0x7f5fe4068990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f600415ac70 0x7f600414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f60040c8eb0 0x7f600414d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 --2- 172.21.15.8:0/452242594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f600406aeb0 0x7f60040c5240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.781+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 >> 172.21.15.8:0/452242594 conn(0x7f60040cef00 msgr2=0x7f6004157bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.785+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 shutdown_connections 2023-10-30T12:45:19.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.785+0000 7f600359e640 1 -- 172.21.15.8:0/452242594 wait complete. 2023-10-30T12:45:19.805 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-30T12:45:19.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 Processor -- start 2023-10-30T12:45:19.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 -- start start 2023-10-30T12:45:19.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec153ba0 0x7fdfec0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdfec0c4410 0x7fdfec15a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec153f80 con 0x7fdfec0c4810 2023-10-30T12:45:19.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec159df0 con 0x7fdfec0c4410 2023-10-30T12:45:19.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff3c6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec15a7c0 con 0x7fdfec153ba0 2023-10-30T12:45:19.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdfec0c4410 0x7fdfec15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdfec0c4410 0x7fdfec15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35956/0 (socket says 172.21.15.8:35956) 2023-10-30T12:45:19.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 -- 172.21.15.8:0/3658937974 learned_addr learned my addr 172.21.15.8:0/3658937974 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:19.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1a07640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec153ba0 0x7fdfec0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 -- 172.21.15.8:0/3658937974 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec153ba0 msgr2=0x7fdfec0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec153ba0 0x7fdfec0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 -- 172.21.15.8:0/3658937974 >> v1:172.21.15.46:6789/0 conn(0x7fdfec0c4810 legacy=0x7fdfec15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.949+0000 7fdff1206640 1 -- 172.21.15.8:0/3658937974 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec1597c0 con 0x7fdfec0c4410 2023-10-30T12:45:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff1206640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec0c4410 0x7fdfec15a280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4f262fc56eb943 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff0a05640 1 -- 172.21.15.8:0/3658937974 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0ab680 con 0x7fdfec0c4410 2023-10-30T12:45:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff0a05640 1 -- 172.21.15.8:0/3658937974 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc0ab7e0 con 0x7fdfec0c4410 2023-10-30T12:45:19.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff0a05640 1 -- 172.21.15.8:0/3658937974 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0abb70 con 0x7fdfec0c4410 2023-10-30T12:45:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 -- 172.21.15.8:0/3658937974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec0c4410 msgr2=0x7fdfec15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec0c4410 0x7fdfec15a280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc01c480 comp rx=0 tx=0).stop 2023-10-30T12:45:19.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 -- 172.21.15.8:0/3658937974 shutdown_connections 2023-10-30T12:45:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec0c4410 0x7fdfec15a280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/3658937974 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec153ba0 0x7fdfec0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 -- 172.21.15.8:0/3658937974 >> 172.21.15.8:0/3658937974 conn(0x7fdfec071790 msgr2=0x7fdfec071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 -- 172.21.15.8:0/3658937974 shutdown_connections 2023-10-30T12:45:19.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.953+0000 7fdff3c6b640 1 -- 172.21.15.8:0/3658937974 wait complete. 2023-10-30T12:45:19.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 Processor -- start 2023-10-30T12:45:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- start start 2023-10-30T12:45:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec0c4410 0x7fdfec1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfec0c4810 0x7fdfec0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b3c80 con 0x7fdfec153ba0 2023-10-30T12:45:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b5120 con 0x7fdfec0c4810 2023-10-30T12:45:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b3700 con 0x7fdfec0c4410 2023-10-30T12:45:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35962/0 (socket says 172.21.15.8:35962) 2023-10-30T12:45:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 -- 172.21.15.8:0/779713713 learned_addr learned my addr 172.21.15.8:0/779713713 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff1206640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfec0c4810 0x7fdfec0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff1a07640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec0c4410 0x7fdfec1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 -- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec0c4410 msgr2=0x7fdfec1f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec0c4410 0x7fdfec1f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 -- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfec0c4810 msgr2=0x7fdfec0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfec0c4810 0x7fdfec0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 -- 172.21.15.8:0/779713713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc0094d0 con 0x7fdfec153ba0 2023-10-30T12:45:19.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff2208640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c7a0 tx=0x7fdfe400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4003b80 con 0x7fdfec153ba0 2023-10-30T12:45:19.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4003ce0 con 0x7fdfec153ba0 2023-10-30T12:45:19.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec0b4b60 con 0x7fdfec153ba0 2023-10-30T12:45:19.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400aa30 con 0x7fdfec153ba0 2023-10-30T12:45:19.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.957+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec0b39a0 con 0x7fdfec153ba0 2023-10-30T12:45:19.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec04e170 con 0x7fdfec153ba0 2023-10-30T12:45:19.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdfe400ab90 con 0x7fdfec153ba0 2023-10-30T12:45:19.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdfe27fc640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 0x7fdfd0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:19.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfe4083ce0 con 0x7fdfec153ba0 2023-10-30T12:45:19.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdff1a07640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 0x7fdfd0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:19.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.961+0000 7fdff1a07640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 0x7fdfd0068990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b930 tx=0x7fdfe8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:19.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:19.965+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdfe40510c0 con 0x7fdfec153ba0 2023-10-30T12:45:20.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:20.157+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdfec0cabb0 con 0x7fdfec153ba0 2023-10-30T12:45:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdfe27fc640 1 -- 172.21.15.8:0/779713713 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdfe40546e0 con 0x7fdfec153ba0 2023-10-30T12:45:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 msgr2=0x7fdfd0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 0x7fdfd0068990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b930 tx=0x7fdfe8008040 comp rx=0 tx=0).stop 2023-10-30T12:45:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 msgr2=0x7fdfec14e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c7a0 tx=0x7fdfe400cc60 comp rx=0 tx=0).stop 2023-10-30T12:45:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 shutdown_connections 2023-10-30T12:45:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdfd00664d0 0x7fdfd0068990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfec0c4810 0x7fdfec0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdfec153ba0 0x7fdfec14e230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 --2- 172.21.15.8:0/779713713 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdfec0c4410 0x7fdfec1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 >> 172.21.15.8:0/779713713 conn(0x7fdfec071790 msgr2=0x7fdfec158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 shutdown_connections 2023-10-30T12:45:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.101+0000 7fdff3c6b640 1 -- 172.21.15.8:0/779713713 wait complete. 2023-10-30T12:45:21.128 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2023-10-30T12:45:21.129 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-30T12:45:21.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 Processor -- start 2023-10-30T12:45:21.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 -- start start 2023-10-30T12:45:21.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67cc0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc153f80 con 0x7f67cc153ba0 2023-10-30T12:45:21.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc159df0 con 0x7f67cc0c4410 2023-10-30T12:45:21.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d24ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1597c0 con 0x7f67cc0c4810 2023-10-30T12:45:21.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67cbfff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f67cc153ba0 0x7f67cc15cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44704/0 (socket says 172.21.15.8:44704) 2023-10-30T12:45:21.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67cbfff640 1 -- 172.21.15.8:0/821206325 learned_addr learned my addr 172.21.15.8:0/821206325 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:21.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67d0a87640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67cc0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745753968 0 0) 0x7f67cc153f80 con 0x7f67cc153ba0 2023-10-30T12:45:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b4003530 con 0x7f67cc153ba0 2023-10-30T12:45:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474346963 0 0) 0x7f67b4003530 con 0x7f67cc153ba0 2023-10-30T12:45:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 msgr2=0x7f67cc0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67cc0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 >> v1:172.21.15.46:6789/0 conn(0x7f67cc0c4410 legacy=0x7f67cc0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc15d580 con 0x7f67cc153ba0 2023-10-30T12:45:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f67bc001680 con 0x7f67cc153ba0 2023-10-30T12:45:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 >> v1:172.21.15.8:6789/0 conn(0x7f67cc153ba0 legacy=0x7f67cc15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40039b0 0x7f67b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4004d10 0x7f67b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1597c0 con 0x7f67b40039b0 2023-10-30T12:45:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b400bac0 con 0x7f67b4004d10 2023-10-30T12:45:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.277+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b400bc80 con 0x7f67cc0c4810 2023-10-30T12:45:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40039b0 0x7f67b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cb7fe640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67d0a87640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4004d10 0x7f67b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 -- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 msgr2=0x7f67b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:21.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67cc0c4810 0x7f67b40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 -- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4004d10 msgr2=0x7f67b4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:21.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4004d10 0x7f67b4005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 -- 172.21.15.8:0/821206325 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4003530 con 0x7f67b40039b0 2023-10-30T12:45:21.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67cbfff640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40039b0 0x7f67b4003dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00c240 tx=0x7f67bc01cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f873c0330ce3a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:21.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc005ae0 con 0x7f67b40039b0 2023-10-30T12:45:21.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc005c40 con 0x7f67b40039b0 2023-10-30T12:45:21.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.281+0000 7f67caffd640 1 -- 172.21.15.8:0/821206325 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc007070 con 0x7f67b40039b0 2023-10-30T12:45:21.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- 172.21.15.8:0/821206325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40039b0 msgr2=0x7f67b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 --2- 172.21.15.8:0/821206325 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40039b0 0x7f67b4003dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00c240 tx=0x7f67bc01cb50 comp rx=0 tx=0).stop 2023-10-30T12:45:21.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d0a87640 1 -- 172.21.15.8:0/821206325 reap_dead start 2023-10-30T12:45:21.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- 172.21.15.8:0/821206325 shutdown_connections 2023-10-30T12:45:21.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- 172.21.15.8:0/821206325 >> 172.21.15.8:0/821206325 conn(0x7f67cc071790 msgr2=0x7f67cc071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:21.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- 172.21.15.8:0/821206325 shutdown_connections 2023-10-30T12:45:21.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- 172.21.15.8:0/821206325 wait complete. 2023-10-30T12:45:21.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 Processor -- start 2023-10-30T12:45:21.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- start start 2023-10-30T12:45:21.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b40039b0 0x7f67cc1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc1f4550 0x7f67cc1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc154590 con 0x7f67cc154130 2023-10-30T12:45:21.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1f4f20 con 0x7f67cc1f4550 2023-10-30T12:45:21.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d24ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1f34d0 con 0x7f67b40039b0 2023-10-30T12:45:21.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35988/0 (socket says 172.21.15.8:35988) 2023-10-30T12:45:21.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 -- 172.21.15.8:0/2650361931 learned_addr learned my addr 172.21.15.8:0/2650361931 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:21.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cbfff640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b40039b0 0x7f67cc1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67d0a87640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc1f4550 0x7f67cc1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 -- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b40039b0 msgr2=0x7f67cc1f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b40039b0 0x7f67cc1f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 -- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc1f4550 msgr2=0x7f67cc1f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:21.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc1f4550 0x7f67cc1f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:21.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 -- 172.21.15.8:0/2650361931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc15a7f0 con 0x7f67cc154130 2023-10-30T12:45:21.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.285+0000 7f67cb7fe640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e550 tx=0x7f67c000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:21.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00052e0 con 0x7f67cc154130 2023-10-30T12:45:21.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0005440 con 0x7f67cc154130 2023-10-30T12:45:21.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc1f37b0 con 0x7f67cc154130 2023-10-30T12:45:21.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0003820 con 0x7f67cc154130 2023-10-30T12:45:21.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc1f3d70 con 0x7f67cc154130 2023-10-30T12:45:21.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc04e170 con 0x7f67cc154130 2023-10-30T12:45:21.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67c0003980 con 0x7f67cc154130 2023-10-30T12:45:21.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 0x7f67b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:21.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67c0083c80 con 0x7f67cc154130 2023-10-30T12:45:21.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67cbfff640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 0x7f67b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:21.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.289+0000 7f67cbfff640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 0x7f67b8068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ab000 tx=0x7f67bc0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:21.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.293+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67c0051060 con 0x7f67cc154130 2023-10-30T12:45:21.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:21.485+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f67cc14dc30 con 0x7f67cc154130 2023-10-30T12:45:22.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67c8ff9640 1 -- 172.21.15.8:0/2650361931 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f67c0054680 con 0x7f67cc154130 2023-10-30T12:45:22.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 msgr2=0x7f67b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 0x7f67b8068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ab000 tx=0x7f67bc0a7070 comp rx=0 tx=0).stop 2023-10-30T12:45:22.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 msgr2=0x7f67cc155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e550 tx=0x7f67c000ea10 comp rx=0 tx=0).stop 2023-10-30T12:45:22.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 shutdown_connections 2023-10-30T12:45:22.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67b80664a0 0x7f67b8068960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc1f4550 0x7f67cc1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67cc154130 0x7f67cc155d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 --2- 172.21.15.8:0/2650361931 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b40039b0 0x7f67cc1f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.109+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 >> 172.21.15.8:0/2650361931 conn(0x7f67cc071790 msgr2=0x7f67cc14de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:22.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.113+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 shutdown_connections 2023-10-30T12:45:22.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.113+0000 7f67d24ea640 1 -- 172.21.15.8:0/2650361931 wait complete. 2023-10-30T12:45:22.136 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-30T12:45:22.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 Processor -- start 2023-10-30T12:45:22.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 -- start start 2023-10-30T12:45:22.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f40c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 0x7fa7f4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f4157100 con 0x7fa7f40c1430 2023-10-30T12:45:22.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f4150a10 con 0x7fa7f40c5be0 2023-10-30T12:45:22.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7fa7c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4072000 con 0x7fa7f4150630 2023-10-30T12:45:22.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 0x7fa7f4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f8d62640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7f40c1430 0x7fa7f40cacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:32966/0 (socket says 172.21.15.8:32966) 2023-10-30T12:45:22.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f40c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f8d62640 1 -- 172.21.15.8:0/271124040 learned_addr learned my addr 172.21.15.8:0/271124040 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:22.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f3fff640 1 -- 172.21.15.8:0/271124040 >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 msgr2=0x7fa7f4159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:22.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f3fff640 1 --2- 172.21.15.8:0/271124040 >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 0x7fa7f4159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f3fff640 1 -- 172.21.15.8:0/271124040 >> v1:172.21.15.46:6789/0 conn(0x7fa7f40c1430 legacy=0x7fa7f40cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f3fff640 1 -- 172.21.15.8:0/271124040 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f4156510 con 0x7fa7f4150630 2023-10-30T12:45:22.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.281+0000 7fa7f37fe640 1 --2- 172.21.15.8:0/271124040 >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 0x7fa7f4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:22.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7f3fff640 1 --2- 172.21.15.8:0/271124040 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f40c56a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40097c0 tx=0x7fa7e401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6018b8f8edce4c25 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:22.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7f2ffd640 1 -- 172.21.15.8:0/271124040 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40ab600 con 0x7fa7f4150630 2023-10-30T12:45:22.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7f2ffd640 1 -- 172.21.15.8:0/271124040 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e40a7050 con 0x7fa7f4150630 2023-10-30T12:45:22.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7f2ffd640 1 -- 172.21.15.8:0/271124040 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40ab980 con 0x7fa7f4150630 2023-10-30T12:45:22.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/271124040 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 msgr2=0x7fa7f40c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/271124040 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f40c56a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40097c0 tx=0x7fa7e401c3f0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/271124040 shutdown_connections 2023-10-30T12:45:22.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/271124040 >> v2:172.21.15.46:3300/0 conn(0x7fa7f40c5be0 0x7fa7f4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/271124040 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f40c56a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/271124040 >> 172.21.15.8:0/271124040 conn(0x7fa7f406da40 msgr2=0x7fa7f406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:22.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/271124040 shutdown_connections 2023-10-30T12:45:22.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.285+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/271124040 wait complete. 2023-10-30T12:45:22.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 Processor -- start 2023-10-30T12:45:22.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- start start 2023-10-30T12:45:22.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7f40c1430 0x7fa7f41f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f4142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41f5c20 con 0x7fa7f40c5be0 2023-10-30T12:45:22.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4158100 con 0x7fa7f40c1430 2023-10-30T12:45:22.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41584d0 con 0x7fa7f4150630 2023-10-30T12:45:22.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35992/0 (socket says 172.21.15.8:35992) 2023-10-30T12:45:22.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 -- 172.21.15.8:0/188567178 learned_addr learned my addr 172.21.15.8:0/188567178 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:22.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f8d62640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f3fff640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7f40c1430 0x7fa7f41f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 -- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 msgr2=0x7fa7f4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f4142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 -- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7f40c1430 msgr2=0x7fa7f41f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:22.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7f40c1430 0x7fa7f41f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:22.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 -- 172.21.15.8:0/188567178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8009420 con 0x7fa7f40c5be0 2023-10-30T12:45:22.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f37fe640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ebe0 tx=0x7fa7e800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:22.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800eee0 con 0x7fa7f40c5be0 2023-10-30T12:45:22.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e800cbd0 con 0x7fa7f40c5be0 2023-10-30T12:45:22.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4009420 con 0x7fa7f40c5be0 2023-10-30T12:45:22.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8013470 con 0x7fa7f40c5be0 2023-10-30T12:45:22.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.289+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f40c9df0 con 0x7fa7f40c5be0 2023-10-30T12:45:22.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f41f5d60 con 0x7fa7f40c5be0 2023-10-30T12:45:22.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa7e80135d0 con 0x7fa7f40c5be0 2023-10-30T12:45:22.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7f0ff9640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 0x7fa7e0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:22.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7e8083fe0 con 0x7fa7f40c5be0 2023-10-30T12:45:22.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7f3fff640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 0x7fa7e0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:22.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.293+0000 7fa7f3fff640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 0x7fa7e0068990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40ad000 tx=0x7fa7e40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:22.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.297+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7f41f5d60 con 0x7fa7f40c5be0 2023-10-30T12:45:22.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:22.489+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa7f41f5d60 con 0x7fa7f40c5be0 2023-10-30T12:45:23.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.109+0000 7fa7f0ff9640 1 -- 172.21.15.8:0/188567178 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7f41f5d60 con 0x7fa7f40c5be0 2023-10-30T12:45:23.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 msgr2=0x7fa7e0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 0x7fa7e0068990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40ad000 tx=0x7fa7e40a6000 comp rx=0 tx=0).stop 2023-10-30T12:45:23.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 msgr2=0x7fa7f40ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ebe0 tx=0x7fa7e800c520 comp rx=0 tx=0).stop 2023-10-30T12:45:23.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 shutdown_connections 2023-10-30T12:45:23.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7e00664d0 0x7fa7e0068990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7f4150630 0x7fa7f4142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7f40c5be0 0x7fa7f40ca270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 --2- 172.21.15.8:0/188567178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7f40c1430 0x7fa7f41f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 >> 172.21.15.8:0/188567178 conn(0x7fa7f406da40 msgr2=0x7fa7f40721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:23.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 shutdown_connections 2023-10-30T12:45:23.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.113+0000 7fa7fa7c5640 1 -- 172.21.15.8:0/188567178 wait complete. 2023-10-30T12:45:23.141 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-30T12:45:23.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.285+0000 7f1d9a0c9640 1 Processor -- start 2023-10-30T12:45:23.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.285+0000 7f1d9a0c9640 1 -- start start 2023-10-30T12:45:23.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d9a0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d940cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d9a0c9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d940c1020 0x7f1d94072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d9a0c9640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d941581f0 con 0x7f1d940725c0 2023-10-30T12:45:23.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d9a0c9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d941576d0 con 0x7f1d940c1020 2023-10-30T12:45:23.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d9a0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94158680 con 0x7f1d94151a20 2023-10-30T12:45:23.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d93fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f1d940725c0 0x7f1d940c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44706/0 (socket says 172.21.15.8:44706) 2023-10-30T12:45:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d93fff640 1 -- 172.21.15.8:0/4143299317 learned_addr learned my addr 172.21.15.8:0/4143299317 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d92ffd640 1 --2- 172.21.15.8:0/4143299317 >> v2:172.21.15.46:3300/0 conn(0x7f1d940c1020 0x7f1d94072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d940cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771788427 0 0) 0x7f1d941581f0 con 0x7f1d940725c0 2023-10-30T12:45:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d7c003530 con 0x7f1d940725c0 2023-10-30T12:45:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242435908 0 0) 0x7f1d7c003530 con 0x7f1d940725c0 2023-10-30T12:45:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 msgr2=0x7f1d940cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d940cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 >> v2:172.21.15.46:3300/0 conn(0x7f1d940c1020 msgr2=0x7f1d94072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 --2- 172.21.15.8:0/4143299317 >> v2:172.21.15.46:3300/0 conn(0x7f1d940c1020 0x7f1d94072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940cb9f0 con 0x7f1d940725c0 2023-10-30T12:45:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f1d8c001a00 con 0x7f1d940725c0 2023-10-30T12:45:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 >> v1:172.21.15.8:6789/0 conn(0x7f1d940725c0 legacy=0x7f1d940c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d7c0039b0 0x7f1d7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d7c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d7c004d10 0x7f1d7c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94158680 con 0x7f1d7c0039b0 2023-10-30T12:45:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c00b990 con 0x7f1d7c004d10 2023-10-30T12:45:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c00bad0 con 0x7f1d94151a20 2023-10-30T12:45:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d7c0039b0 0x7f1d7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d92ffd640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d7c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d93fff640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d7c004d10 0x7f1d7c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 -- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 msgr2=0x7f1d7c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d94151a20 0x7f1d7c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 -- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d7c004d10 msgr2=0x7f1d7c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d7c004d10 0x7f1d7c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d937fe640 1 -- 172.21.15.8:0/4143299317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c003530 con 0x7f1d7c0039b0 2023-10-30T12:45:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.289+0000 7f1d93fff640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d7c004d10 0x7f1d7c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:23.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d937fe640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d7c0039b0 0x7f1d7c003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ae0 tx=0x7f1d8401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7db817378b55643 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:23.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840a5640 con 0x7f1d7c0039b0 2023-10-30T12:45:23.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c002100 con 0x7f1d7c0039b0 2023-10-30T12:45:23.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d927fc640 1 -- 172.21.15.8:0/4143299317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840a59e0 con 0x7f1d7c0039b0 2023-10-30T12:45:23.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/4143299317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d7c0039b0 msgr2=0x7f1d7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/4143299317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d7c0039b0 0x7f1d7c003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009ae0 tx=0x7f1d8401ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:23.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d93fff640 1 -- 172.21.15.8:0/4143299317 reap_dead start 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/4143299317 shutdown_connections 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/4143299317 >> 172.21.15.8:0/4143299317 conn(0x7f1d9406da40 msgr2=0x7f1d9406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/4143299317 shutdown_connections 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/4143299317 wait complete. 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.293+0000 7f1d9a0c9640 1 Processor -- start 2023-10-30T12:45:23.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- start start 2023-10-30T12:45:23.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d7c0039b0 0x7f1d94156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 0x7f1d94146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94070830 con 0x7f1d9414d230 2023-10-30T12:45:23.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9414da40 con 0x7f1d940703b0 2023-10-30T12:45:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9414d7a0 con 0x7f1d7c0039b0 2023-10-30T12:45:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36012/0 (socket says 172.21.15.8:36012) 2023-10-30T12:45:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 -- 172.21.15.8:0/703093744 learned_addr learned my addr 172.21.15.8:0/703093744 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d937fe640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d7c0039b0 0x7f1d94156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d93fff640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 0x7f1d94146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 -- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d7c0039b0 msgr2=0x7f1d94156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d7c0039b0 0x7f1d94156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 -- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 msgr2=0x7f1d94146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:23.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 0x7f1d94146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 -- 172.21.15.8:0/703093744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d941e5b60 con 0x7f1d9414d230 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d93fff640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 0x7f1d94146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d92ffd640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d88011660 tx=0x7f1d88011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800f9c0 con 0x7f1d9414d230 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800fb20 con 0x7f1d9414d230 2023-10-30T12:45:23.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d941e5e40 con 0x7f1d9414d230 2023-10-30T12:45:23.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800fe90 con 0x7f1d9414d230 2023-10-30T12:45:23.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d941e6400 con 0x7f1d9414d230 2023-10-30T12:45:23.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.297+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d941e65c0 con 0x7f1d9414d230 2023-10-30T12:45:23.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.301+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d88015460 con 0x7f1d9414d230 2023-10-30T12:45:23.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.301+0000 7f1d7bfff640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 0x7f1d80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:23.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.301+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d88084300 con 0x7f1d9414d230 2023-10-30T12:45:23.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.301+0000 7f1d937fe640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 0x7f1d80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:23.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.301+0000 7f1d937fe640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 0x7f1d80068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d840046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:23.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.305+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d880516f0 con 0x7f1d9414d230 2023-10-30T12:45:23.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:23.493+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1d9404e170 con 0x7f1d9414d230 2023-10-30T12:45:24.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d7bfff640 1 -- 172.21.15.8:0/703093744 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1d88054d10 con 0x7f1d9414d230 2023-10-30T12:45:24.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 msgr2=0x7f1d80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 0x7f1d80068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d840046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 msgr2=0x7f1d9406fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d88011660 tx=0x7f1d88011b20 comp rx=0 tx=0).stop 2023-10-30T12:45:24.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 shutdown_connections 2023-10-30T12:45:24.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d800664a0 0x7f1d80068960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d940703b0 0x7f1d94146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d9414d230 0x7f1d9406fe70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 --2- 172.21.15.8:0/703093744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d7c0039b0 0x7f1d94156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 >> 172.21.15.8:0/703093744 conn(0x7f1d9406da40 msgr2=0x7f1d9406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:24.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 shutdown_connections 2023-10-30T12:45:24.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.129+0000 7f1d9a0c9640 1 -- 172.21.15.8:0/703093744 wait complete. 2023-10-30T12:45:24.156 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-30T12:45:24.157 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:24.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 Processor -- start 2023-10-30T12:45:24.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 -- start start 2023-10-30T12:45:24.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f457815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45780c42a0 0x7f45780ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4578159f20 con 0x7f45780c4a10 2023-10-30T12:45:24.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457815dec0 con 0x7f45780c42a0 2023-10-30T12:45:24.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457f26b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45781599a0 con 0x7f4578153c80 2023-10-30T12:45:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457c806640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f45780c4a10 0x7f457815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44714/0 (socket says 172.21.15.8:44714) 2023-10-30T12:45:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457c806640 1 -- 172.21.15.8:0/2948658847 learned_addr learned my addr 172.21.15.8:0/2948658847 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457d007640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f457815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457d808640 1 --2- 172.21.15.8:0/2948658847 >> v2:172.21.15.46:3300/0 conn(0x7f45780c42a0 0x7f45780ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553196043 0 0) 0x7f4578159f20 con 0x7f45780c4a10 2023-10-30T12:45:24.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4564003530 con 0x7f45780c4a10 2023-10-30T12:45:24.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204698050 0 0) 0x7f4564003530 con 0x7f45780c4a10 2023-10-30T12:45:24.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 msgr2=0x7f457815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f457815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 >> v2:172.21.15.46:3300/0 conn(0x7f45780c42a0 msgr2=0x7f45780ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 --2- 172.21.15.8:0/2948658847 >> v2:172.21.15.46:3300/0 conn(0x7f45780c42a0 0x7f45780ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457815d620 con 0x7f45780c4a10 2023-10-30T12:45:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f4568001680 con 0x7f45780c4a10 2023-10-30T12:45:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 >> v1:172.21.15.8:6789/0 conn(0x7f45780c4a10 legacy=0x7f457815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f4564003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45640047d0 0x7f4564004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f4564005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457815dec0 con 0x7f4564003980 2023-10-30T12:45:24.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781599a0 con 0x7f45640047d0 2023-10-30T12:45:24.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456400b990 con 0x7f4578153c80 2023-10-30T12:45:24.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457d007640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f4564003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457d808640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f4564005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.301+0000 7f457c806640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45640047d0 0x7f4564004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 -- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 msgr2=0x7f4564005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578153c80 0x7f4564005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 -- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45640047d0 msgr2=0x7f4564004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45640047d0 0x7f4564004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 -- 172.21.15.8:0/2948658847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564003530 con 0x7f4564003980 2023-10-30T12:45:24.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d007640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f4564003da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4574009ae0 tx=0x7f457401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e59e9b428b7990 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740a6460 con 0x7f4564003980 2023-10-30T12:45:24.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45740a65c0 con 0x7f4564003980 2023-10-30T12:45:24.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f456ffff640 1 -- 172.21.15.8:0/2948658847 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740a6970 con 0x7f4564003980 2023-10-30T12:45:24.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457f26b640 1 -- 172.21.15.8:0/2948658847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 msgr2=0x7f4564003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457f26b640 1 --2- 172.21.15.8:0/2948658847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f4564003da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4574009ae0 tx=0x7f457401ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:24.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457d808640 1 -- 172.21.15.8:0/2948658847 reap_dead start 2023-10-30T12:45:24.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457f26b640 1 -- 172.21.15.8:0/2948658847 shutdown_connections 2023-10-30T12:45:24.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.305+0000 7f457f26b640 1 -- 172.21.15.8:0/2948658847 >> 172.21.15.8:0/2948658847 conn(0x7f45780715c0 msgr2=0x7f45780719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:24.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- 172.21.15.8:0/2948658847 shutdown_connections 2023-10-30T12:45:24.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- 172.21.15.8:0/2948658847 wait complete. 2023-10-30T12:45:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 Processor -- start 2023-10-30T12:45:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- start start 2023-10-30T12:45:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 0x7f4578146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 0x7f457815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781474b0 con 0x7f4564003980 2023-10-30T12:45:24.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781ef6a0 con 0x7f4578152250 2023-10-30T12:45:24.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457f26b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45781526d0 con 0x7f4578150680 2023-10-30T12:45:24.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36034/0 (socket says 172.21.15.8:36034) 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 -- 172.21.15.8:0/1185831186 learned_addr learned my addr 172.21.15.8:0/1185831186 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d808640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 0x7f457815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457c806640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 0x7f4578146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 -- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 msgr2=0x7f457815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 0x7f457815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 -- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 msgr2=0x7f4578146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 0x7f4578146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 -- 172.21.15.8:0/1185831186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780c5f00 con 0x7f4564003980 2023-10-30T12:45:24.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d808640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 0x7f457815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:24.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457c806640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 0x7f4578146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:24.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.309+0000 7f457d007640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4574009ae0 tx=0x7f4564005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740a7440 con 0x7f4564003980 2023-10-30T12:45:24.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45740a75a0 con 0x7f4564003980 2023-10-30T12:45:24.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740a7930 con 0x7f4564003980 2023-10-30T12:45:24.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457815b700 con 0x7f4564003980 2023-10-30T12:45:24.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45781ef050 con 0x7f4564003980 2023-10-30T12:45:24.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457804e170 con 0x7f4564003980 2023-10-30T12:45:24.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45740a7a90 con 0x7f4564003980 2023-10-30T12:45:24.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 0x7f455c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f457411ded0 con 0x7f4564003980 2023-10-30T12:45:24.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.313+0000 7f457c806640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 0x7f455c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.317+0000 7f457c806640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 0x7f455c068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c10 tx=0x7f45680047f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.317+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45740eb220 con 0x7f4564003980 2023-10-30T12:45:24.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.505+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4578159b80 con 0x7f4564003980 2023-10-30T12:45:24.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.505+0000 7f456dffb640 1 -- 172.21.15.8:0/1185831186 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6492 (secure 0 0 0) 0x7f45740be030 con 0x7f4564003980 2023-10-30T12:45:24.509 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:24.510 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:24.133418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4303,"mds_1":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:24.510 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 9 2023-10-30T12:45:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 msgr2=0x7f455c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 0x7f455c068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c10 tx=0x7f45680047f0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 msgr2=0x7f45781efb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4574009ae0 tx=0x7f4564005800 comp rx=0 tx=0).stop 2023-10-30T12:45:24.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 shutdown_connections 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f455c0664a0 0x7f455c068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4578152250 0x7f4578146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4578150680 0x7f457815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 --2- 172.21.15.8:0/1185831186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4564003980 0x7f45781efb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 >> 172.21.15.8:0/1185831186 conn(0x7f45780715c0 msgr2=0x7f45780cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:24.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 shutdown_connections 2023-10-30T12:45:24.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.509+0000 7f457f26b640 1 -- 172.21.15.8:0/1185831186 wait complete. 2023-10-30T12:45:24.531 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:24.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 Processor -- start 2023-10-30T12:45:24.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 -- start start 2023-10-30T12:45:24.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f115c0c4a10 0x7f115c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f115c15a3b0 0x7f115c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c15dec0 con 0x7f115c15a3b0 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c1599a0 con 0x7f115c0c4a10 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f116246b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115c159f20 con 0x7f115c153c80 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f115c15a3b0 0x7f115c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f115c15a3b0 0x7f115c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36050/0 (socket says 172.21.15.8:36050) 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 -- 172.21.15.8:0/2531147679 learned_addr learned my addr 172.21.15.8:0/2531147679 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:24.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f115b7fe640 1 --2- 172.21.15.8:0/2531147679 >> v2:172.21.15.46:3300/0 conn(0x7f115c0c4a10 0x7f115c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f115bfff640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 -- 172.21.15.8:0/2531147679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 msgr2=0x7f115c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 -- 172.21.15.8:0/2531147679 >> v2:172.21.15.46:3300/0 conn(0x7f115c0c4a10 msgr2=0x7f115c0c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:24.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 --2- 172.21.15.8:0/2531147679 >> v2:172.21.15.46:3300/0 conn(0x7f115c0c4a10 0x7f115c0c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.673+0000 7f1160a08640 1 -- 172.21.15.8:0/2531147679 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0c47e0 con 0x7f115c15a3b0 2023-10-30T12:45:24.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f1160a08640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c15a3b0 0x7f115c0ca1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f115c08cc20 tx=0x7f115401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2560b030481c2783 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f115affd640 1 -- 172.21.15.8:0/2531147679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540af400 con 0x7f115c15a3b0 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f115affd640 1 -- 172.21.15.8:0/2531147679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540ab050 con 0x7f115c15a3b0 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f115affd640 1 -- 172.21.15.8:0/2531147679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540af780 con 0x7f115c15a3b0 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 -- 172.21.15.8:0/2531147679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c15a3b0 msgr2=0x7f115c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c15a3b0 0x7f115c0ca1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f115c08cc20 tx=0x7f115401f1c0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 -- 172.21.15.8:0/2531147679 shutdown_connections 2023-10-30T12:45:24.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c15a3b0 0x7f115c0ca1e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 --2- 172.21.15.8:0/2531147679 >> v2:172.21.15.46:3300/0 conn(0x7f115c0c4a10 0x7f115c0c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 --2- 172.21.15.8:0/2531147679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 -- 172.21.15.8:0/2531147679 >> 172.21.15.8:0/2531147679 conn(0x7f115c0715c0 msgr2=0x7f115c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:24.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 -- 172.21.15.8:0/2531147679 shutdown_connections 2023-10-30T12:45:24.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.677+0000 7f116246b640 1 -- 172.21.15.8:0/2531147679 wait complete. 2023-10-30T12:45:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 Processor -- start 2023-10-30T12:45:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 -- start start 2023-10-30T12:45:24.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c1ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115c15a3b0 0x7f115c14f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c14ad60 con 0x7f115c0c4a10 2023-10-30T12:45:24.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1ef000 con 0x7f115c15a3b0 2023-10-30T12:45:24.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0ce3b0 con 0x7f115c153c80 2023-10-30T12:45:24.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36058/0 (socket says 172.21.15.8:36058) 2023-10-30T12:45:24.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 -- 172.21.15.8:0/2630169874 learned_addr learned my addr 172.21.15.8:0/2630169874 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:24.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f1160a08640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115c15a3b0 0x7f115c14f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115b7fe640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c1ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 -- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 msgr2=0x7f115c1ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c1ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 -- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115c15a3b0 msgr2=0x7f115c14f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115c15a3b0 0x7f115c14f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 -- 172.21.15.8:0/2630169874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009b30 con 0x7f115c0c4a10 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f115bfff640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d800 tx=0x7f114c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c00b6c0 con 0x7f115c0c4a10 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c00b820 con 0x7f115c0c4a10 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c00bb90 con 0x7f115c0c4a10 2023-10-30T12:45:24.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.681+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1f2f20 con 0x7f115c0c4a10 2023-10-30T12:45:24.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1f2960 con 0x7f115c0c4a10 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c04e170 con 0x7f115c0c4a10 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f114c00bcf0 con 0x7f115c0c4a10 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f1158ff9640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 0x7f1148068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f114c084580 con 0x7f115c0c4a10 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f115b7fe640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 0x7f1148068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:24.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.685+0000 7f115b7fe640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 0x7f1148068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f115c158b60 tx=0x7f1150008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:24.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.689+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f114c0518d0 con 0x7f115c0c4a10 2023-10-30T12:45:24.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.849+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f115c0cad40 con 0x7f115c0c4a10 2023-10-30T12:45:24.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.849+0000 7f1158ff9640 1 -- 172.21.15.8:0/2630169874 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f114c054ef0 con 0x7f115c0c4a10 2023-10-30T12:45:24.854 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:24.854 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:24.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 msgr2=0x7f1148068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 0x7f1148068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f115c158b60 tx=0x7f1150008040 comp rx=0 tx=0).stop 2023-10-30T12:45:24.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 msgr2=0x7f115c1f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:24.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d800 tx=0x7f114c00dcc0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 shutdown_connections 2023-10-30T12:45:24.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11480664a0 0x7f1148068960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115c15a3b0 0x7f115c14f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f115c153c80 0x7f115c1ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 --2- 172.21.15.8:0/2630169874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f115c0c4a10 0x7f115c1f2030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:24.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 >> 172.21.15.8:0/2630169874 conn(0x7f115c0715c0 msgr2=0x7f115c0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:24.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 shutdown_connections 2023-10-30T12:45:24.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:24.853+0000 7f116246b640 1 -- 172.21.15.8:0/2630169874 wait complete. 2023-10-30T12:45:24.876 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-30T12:45:24.876 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:25.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 Processor -- start 2023-10-30T12:45:25.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 -- start start 2023-10-30T12:45:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 0x7f8ad80c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8ad80c7170 0x7f8ad8159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80c5f80 con 0x7f8ad8153bb0 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad80c7580 con 0x7f8ad80c7170 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adef0f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad80c5690 con 0x7f8ad806c420 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8add4ac640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8ad80c7170 0x7f8ad8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adccab640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8ad8153bb0 0x7f8ad80c6410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44724/0 (socket says 172.21.15.8:44724) 2023-10-30T12:45:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8adccab640 1 -- 172.21.15.8:0/3767721265 learned_addr learned my addr 172.21.15.8:0/3767721265 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/3767721265 >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 0x7f8ad80c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8add4ac640 1 -- 172.21.15.8:0/3767721265 >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 msgr2=0x7f8ad80c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8add4ac640 1 --2- 172.21.15.8:0/3767721265 >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 0x7f8ad80c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8add4ac640 1 -- 172.21.15.8:0/3767721265 >> v1:172.21.15.8:6789/0 conn(0x7f8ad8153bb0 legacy=0x7f8ad80c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8add4ac640 1 -- 172.21.15.8:0/3767721265 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad815a580 con 0x7f8ad80c7170 2023-10-30T12:45:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8ad77fe640 1 -- 172.21.15.8:0/3767721265 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748613960 0 0) 0x7f8ad80c5f80 con 0x7f8ad8153bb0 2023-10-30T12:45:25.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.017+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/3767721265 >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 0x7f8ad80c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8add4ac640 1 --2- 172.21.15.8:0/3767721265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad80c7170 0x7f8ad8159980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ad806f800 tx=0x7f8ad001f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53debc0338d855 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8ad77fe640 1 -- 172.21.15.8:0/3767721265 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0440 con 0x7f8ad80c7170 2023-10-30T12:45:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8ad77fe640 1 -- 172.21.15.8:0/3767721265 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00b05a0 con 0x7f8ad80c7170 2023-10-30T12:45:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8ad77fe640 1 -- 172.21.15.8:0/3767721265 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0930 con 0x7f8ad80c7170 2023-10-30T12:45:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 -- 172.21.15.8:0/3767721265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad80c7170 msgr2=0x7f8ad8159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 --2- 172.21.15.8:0/3767721265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad80c7170 0x7f8ad8159980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ad806f800 tx=0x7f8ad001f0f0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 -- 172.21.15.8:0/3767721265 shutdown_connections 2023-10-30T12:45:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 --2- 172.21.15.8:0/3767721265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad80c7170 0x7f8ad8159980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 --2- 172.21.15.8:0/3767721265 >> v2:172.21.15.46:3300/0 conn(0x7f8ad806c420 0x7f8ad80c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 -- 172.21.15.8:0/3767721265 >> 172.21.15.8:0/3767721265 conn(0x7f8ad80cac70 msgr2=0x7f8ad80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 -- 172.21.15.8:0/3767721265 shutdown_connections 2023-10-30T12:45:25.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.021+0000 7f8adef0f640 1 -- 172.21.15.8:0/3767721265 wait complete. 2023-10-30T12:45:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 Processor -- start 2023-10-30T12:45:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- start start 2023-10-30T12:45:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 0x7f8ad81efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ad8153bb0 0x7f8ad81f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad814df70 con 0x7f8ad806c420 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81562d0 con 0x7f8ad80c7170 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81556c0 con 0x7f8ad8153bb0 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36082/0 (socket says 172.21.15.8:36082) 2023-10-30T12:45:25.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 -- 172.21.15.8:0/2393557794 learned_addr learned my addr 172.21.15.8:0/2393557794 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 0x7f8ad81efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8add4ac640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ad8153bb0 0x7f8ad81f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 -- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ad8153bb0 msgr2=0x7f8ad81f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ad8153bb0 0x7f8ad81f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 -- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 msgr2=0x7f8ad81efaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 0x7f8ad81efaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 -- 172.21.15.8:0/2393557794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009a70 con 0x7f8ad806c420 2023-10-30T12:45:25.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 0x7f8ad81efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adccab640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0011f0 tx=0x7f8acc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0074e0 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc007640 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004a10 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad81559a0 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad8155f60 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.025+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad804e170 con 0x7f8ad806c420 2023-10-30T12:45:25.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.029+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8acc004b70 con 0x7f8ad806c420 2023-10-30T12:45:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.029+0000 7f8ad57fa640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 0x7f8abc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.029+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8acc083270 con 0x7f8ad806c420 2023-10-30T12:45:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.029+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 0x7f8abc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.029+0000 7f8ad7fff640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 0x7f8abc068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8152960 tx=0x7f8ac4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.033+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8acc0505c0 con 0x7f8ad806c420 2023-10-30T12:45:25.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.185+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ad80c5c10 con 0x7f8abc0664a0 2023-10-30T12:45:25.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.193+0000 7f8ad57fa640 1 -- 172.21.15.8:0/2393557794 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264409 (secure 0 0 0) 0x7f8ac400b660 con 0x7f8abc0664a0 2023-10-30T12:45:25.199 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:25.202 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:25.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 msgr2=0x7f8abc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 0x7f8abc068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8152960 tx=0x7f8ac4008040 comp rx=0 tx=0).stop 2023-10-30T12:45:25.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 msgr2=0x7f8ad814da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0011f0 tx=0x7f8acc00c9b0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 shutdown_connections 2023-10-30T12:45:25.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8abc0664a0 0x7f8abc068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80c7170 0x7f8ad81efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ad8153bb0 0x7f8ad81f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 --2- 172.21.15.8:0/2393557794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ad806c420 0x7f8ad814da30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 >> 172.21.15.8:0/2393557794 conn(0x7f8ad80cac70 msgr2=0x7f8ad8070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:25.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 shutdown_connections 2023-10-30T12:45:25.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.201+0000 7f8adef0f640 1 -- 172.21.15.8:0/2393557794 wait complete. 2023-10-30T12:45:25.221 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-30T12:45:23.609604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252068,"kb_used_data":2612,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722652,"statfs":{"total":773094113280,"available":772835995648,"internally_reserved":0,"allocated":2674688,"data_stored":1669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001165"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125671+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:19.125671+0000","last_peered":"2023-10-30T12:45:19.125671+0000","last_clean":"2023-10-30T12:45:19.125671+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:19.125671+0000","last_undegraded":"2023-10-30T12:45:19.125671+0000","last_fullsized":"2023-10-30T12:45:19.125671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:09.263692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.087969+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:19.087969+0000","last_peered":"2023-10-30T12:45:19.087969+0000","last_clean":"2023-10-30T12:45:19.087969+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:19.087969+0000","last_undegraded":"2023-10-30T12:45:19.087969+0000","last_fullsized":"2023-10-30T12:45:19.087969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:01:17.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100587+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:19.100587+0000","last_peered":"2023-10-30T12:45:19.100587+0000","last_clean":"2023-10-30T12:45:19.100587+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:19.100587+0000","last_undegraded":"2023-10-30T12:45:19.100587+0000","last_fullsized":"2023-10-30T12:45:19.100587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:15:42.050956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134450+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:19.134450+0000","last_peered":"2023-10-30T12:45:19.134450+0000","last_clean":"2023-10-30T12:45:19.134450+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:19.134450+0000","last_undegraded":"2023-10-30T12:45:19.134450+0000","last_fullsized":"2023-10-30T12:45:19.134450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:49.883736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125662+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:19.125662+0000","last_peered":"2023-10-30T12:45:19.125662+0000","last_clean":"2023-10-30T12:45:19.125662+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:19.125662+0000","last_undegraded":"2023-10-30T12:45:19.125662+0000","last_fullsized":"2023-10-30T12:45:19.125662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.623327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130904+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:19.130904+0000","last_peered":"2023-10-30T12:45:19.130904+0000","last_clean":"2023-10-30T12:45:19.130904+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:19.130904+0000","last_undegraded":"2023-10-30T12:45:19.130904+0000","last_fullsized":"2023-10-30T12:45:19.130904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.258797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125473+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:19.125473+0000","last_peered":"2023-10-30T12:45:19.125473+0000","last_clean":"2023-10-30T12:45:19.125473+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:19.125473+0000","last_undegraded":"2023-10-30T12:45:19.125473+0000","last_fullsized":"2023-10-30T12:45:19.125473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:04:10.662268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.122905+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:19.122905+0000","last_peered":"2023-10-30T12:45:19.122905+0000","last_clean":"2023-10-30T12:45:19.122905+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:19.122905+0000","last_undegraded":"2023-10-30T12:45:19.122905+0000","last_fullsized":"2023-10-30T12:45:19.122905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:57.965860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.136565+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:19.136565+0000","last_peered":"2023-10-30T12:45:19.136565+0000","last_clean":"2023-10-30T12:45:19.136565+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:19.136565+0000","last_undegraded":"2023-10-30T12:45:19.136565+0000","last_fullsized":"2023-10-30T12:45:19.136565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:36:07.602135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130992+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:19.130992+0000","last_peered":"2023-10-30T12:45:19.130992+0000","last_clean":"2023-10-30T12:45:19.130992+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:19.130992+0000","last_undegraded":"2023-10-30T12:45:19.130992+0000","last_fullsized":"2023-10-30T12:45:19.130992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:11:52.175612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130935+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:19.130935+0000","last_peered":"2023-10-30T12:45:19.130935+0000","last_clean":"2023-10-30T12:45:19.130935+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:19.130935+0000","last_undegraded":"2023-10-30T12:45:19.130935+0000","last_fullsized":"2023-10-30T12:45:19.130935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:57.478243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.120288+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:19.120288+0000","last_peered":"2023-10-30T12:45:19.120288+0000","last_clean":"2023-10-30T12:45:19.120288+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:19.120288+0000","last_undegraded":"2023-10-30T12:45:19.120288+0000","last_fullsized":"2023-10-30T12:45:19.120288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:08.246156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129373+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:19.129373+0000","last_peered":"2023-10-30T12:45:19.129373+0000","last_clean":"2023-10-30T12:45:19.129373+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:19.129373+0000","last_undegraded":"2023-10-30T12:45:19.129373+0000","last_fullsized":"2023-10-30T12:45:19.129373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:47.272376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.098658+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:19.098658+0000","last_peered":"2023-10-30T12:45:19.098658+0000","last_clean":"2023-10-30T12:45:19.098658+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:19.098658+0000","last_undegraded":"2023-10-30T12:45:19.098658+0000","last_fullsized":"2023-10-30T12:45:19.098658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:51:02.515760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.095835+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:19.095835+0000","last_peered":"2023-10-30T12:45:19.095835+0000","last_clean":"2023-10-30T12:45:19.095835+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:19.095835+0000","last_undegraded":"2023-10-30T12:45:19.095835+0000","last_fullsized":"2023-10-30T12:45:19.095835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:29:31.115101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.118982+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:19.118982+0000","last_peered":"2023-10-30T12:45:19.118982+0000","last_clean":"2023-10-30T12:45:19.118982+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:19.118982+0000","last_undegraded":"2023-10-30T12:45:19.118982+0000","last_fullsized":"2023-10-30T12:45:19.118982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:26:59.349423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134339+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:19.134339+0000","last_peered":"2023-10-30T12:45:19.134339+0000","last_clean":"2023-10-30T12:45:19.134339+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:19.134339+0000","last_undegraded":"2023-10-30T12:45:19.134339+0000","last_fullsized":"2023-10-30T12:45:19.134339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:23:45.435309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097633+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:19.097633+0000","last_peered":"2023-10-30T12:45:19.097633+0000","last_clean":"2023-10-30T12:45:19.097633+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:19.097633+0000","last_undegraded":"2023-10-30T12:45:19.097633+0000","last_fullsized":"2023-10-30T12:45:19.097633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:38.269819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125544+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:19.125544+0000","last_peered":"2023-10-30T12:45:19.125544+0000","last_clean":"2023-10-30T12:45:19.125544+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:19.125544+0000","last_undegraded":"2023-10-30T12:45:19.125544+0000","last_fullsized":"2023-10-30T12:45:19.125544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:32.267045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.102834+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:19.102834+0000","last_peered":"2023-10-30T12:45:19.102834+0000","last_clean":"2023-10-30T12:45:19.102834+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:19.102834+0000","last_undegraded":"2023-10-30T12:45:19.102834+0000","last_fullsized":"2023-10-30T12:45:19.102834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:04:53.521179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.122104+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:19.122104+0000","last_peered":"2023-10-30T12:45:19.122104+0000","last_clean":"2023-10-30T12:45:19.122104+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:19.122104+0000","last_undegraded":"2023-10-30T12:45:19.122104+0000","last_fullsized":"2023-10-30T12:45:19.122104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:24:55.744168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134314+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:19.134314+0000","last_peered":"2023-10-30T12:45:19.134314+0000","last_clean":"2023-10-30T12:45:19.134314+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:19.134314+0000","last_undegraded":"2023-10-30T12:45:19.134314+0000","last_fullsized":"2023-10-30T12:45:19.134314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:41:59.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099728+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:19.099728+0000","last_peered":"2023-10-30T12:45:19.099728+0000","last_clean":"2023-10-30T12:45:19.099728+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:19.099728+0000","last_undegraded":"2023-10-30T12:45:19.099728+0000","last_fullsized":"2023-10-30T12:45:19.099728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:41:04.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.118908+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:19.118908+0000","last_peered":"2023-10-30T12:45:19.118908+0000","last_clean":"2023-10-30T12:45:19.118908+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:19.118908+0000","last_undegraded":"2023-10-30T12:45:19.118908+0000","last_fullsized":"2023-10-30T12:45:19.118908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:24:03.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100574+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:19.100574+0000","last_peered":"2023-10-30T12:45:19.100574+0000","last_clean":"2023-10-30T12:45:19.100574+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:19.100574+0000","last_undegraded":"2023-10-30T12:45:19.100574+0000","last_fullsized":"2023-10-30T12:45:19.100574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:57.478243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.115894+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:19.115894+0000","last_peered":"2023-10-30T12:45:19.115894+0000","last_clean":"2023-10-30T12:45:19.115894+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:19.115894+0000","last_undegraded":"2023-10-30T12:45:19.115894+0000","last_fullsized":"2023-10-30T12:45:19.115894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:38:28.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.085908+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:19.085908+0000","last_peered":"2023-10-30T12:45:19.085908+0000","last_clean":"2023-10-30T12:45:19.085908+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:19.085908+0000","last_undegraded":"2023-10-30T12:45:19.085908+0000","last_fullsized":"2023-10-30T12:45:19.085908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:46.957164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099767+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:19.099767+0000","last_peered":"2023-10-30T12:45:19.099767+0000","last_clean":"2023-10-30T12:45:19.099767+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:19.099767+0000","last_undegraded":"2023-10-30T12:45:19.099767+0000","last_fullsized":"2023-10-30T12:45:19.099767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:36.269949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097184+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:19.097184+0000","last_peered":"2023-10-30T12:45:19.097184+0000","last_clean":"2023-10-30T12:45:19.097184+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:19.097184+0000","last_undegraded":"2023-10-30T12:45:19.097184+0000","last_fullsized":"2023-10-30T12:45:19.097184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:05:47.748667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.119876+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:19.119876+0000","last_peered":"2023-10-30T12:45:19.119876+0000","last_clean":"2023-10-30T12:45:19.119876+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:19.119876+0000","last_undegraded":"2023-10-30T12:45:19.119876+0000","last_fullsized":"2023-10-30T12:45:19.119876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:14:05.237539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.108663+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:19.108663+0000","last_peered":"2023-10-30T12:45:19.108663+0000","last_clean":"2023-10-30T12:45:19.108663+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:19.108663+0000","last_undegraded":"2023-10-30T12:45:19.108663+0000","last_fullsized":"2023-10-30T12:45:19.108663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:22.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.101017+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:19.101017+0000","last_peered":"2023-10-30T12:45:19.101017+0000","last_clean":"2023-10-30T12:45:19.101017+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:19.101017+0000","last_undegraded":"2023-10-30T12:45:19.101017+0000","last_fullsized":"2023-10-30T12:45:19.101017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:37:54.417550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.117268+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:19.117268+0000","last_peered":"2023-10-30T12:45:19.117268+0000","last_clean":"2023-10-30T12:45:19.117268+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:19.117268+0000","last_undegraded":"2023-10-30T12:45:19.117268+0000","last_fullsized":"2023-10-30T12:45:19.117268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:43:34.496700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074000+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:19.074000+0000","last_peered":"2023-10-30T12:45:19.074000+0000","last_clean":"2023-10-30T12:45:19.074000+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:19.074000+0000","last_undegraded":"2023-10-30T12:45:19.074000+0000","last_fullsized":"2023-10-30T12:45:19.074000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123644+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:19.123644+0000","last_peered":"2023-10-30T12:45:19.123644+0000","last_clean":"2023-10-30T12:45:19.123644+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:19.123644+0000","last_undegraded":"2023-10-30T12:45:19.123644+0000","last_fullsized":"2023-10-30T12:45:19.123644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:59.171051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071102+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:19.071102+0000","last_peered":"2023-10-30T12:45:19.071102+0000","last_clean":"2023-10-30T12:45:19.071102+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:19.071102+0000","last_undegraded":"2023-10-30T12:45:19.071102+0000","last_fullsized":"2023-10-30T12:45:19.071102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.072044+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:19.072044+0000","last_peered":"2023-10-30T12:45:19.072044+0000","last_clean":"2023-10-30T12:45:19.072044+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:19.072044+0000","last_undegraded":"2023-10-30T12:45:19.072044+0000","last_fullsized":"2023-10-30T12:45:19.072044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.090343+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:19.090343+0000","last_peered":"2023-10-30T12:45:19.090343+0000","last_clean":"2023-10-30T12:45:19.090343+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:19.090343+0000","last_undegraded":"2023-10-30T12:45:19.090343+0000","last_fullsized":"2023-10-30T12:45:19.090343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:45.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.106131+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:19.106131+0000","last_peered":"2023-10-30T12:45:19.106131+0000","last_clean":"2023-10-30T12:45:19.106131+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:19.106131+0000","last_undegraded":"2023-10-30T12:45:19.106131+0000","last_fullsized":"2023-10-30T12:45:19.106131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.085014+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:19.085014+0000","last_peered":"2023-10-30T12:45:19.085014+0000","last_clean":"2023-10-30T12:45:19.085014+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:19.085014+0000","last_undegraded":"2023-10-30T12:45:19.085014+0000","last_fullsized":"2023-10-30T12:45:19.085014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:50.870760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071123+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:19.071123+0000","last_peered":"2023-10-30T12:45:19.071123+0000","last_clean":"2023-10-30T12:45:19.071123+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:19.071123+0000","last_undegraded":"2023-10-30T12:45:19.071123+0000","last_fullsized":"2023-10-30T12:45:19.071123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134275+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:19.134275+0000","last_peered":"2023-10-30T12:45:19.134275+0000","last_clean":"2023-10-30T12:45:19.134275+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:19.134275+0000","last_undegraded":"2023-10-30T12:45:19.134275+0000","last_fullsized":"2023-10-30T12:45:19.134275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:24:47.493636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.113411+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:19.113411+0000","last_peered":"2023-10-30T12:45:19.113411+0000","last_clean":"2023-10-30T12:45:19.113411+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:19.113411+0000","last_undegraded":"2023-10-30T12:45:19.113411+0000","last_fullsized":"2023-10-30T12:45:19.113411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.132142+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:19.132142+0000","last_peered":"2023-10-30T12:45:19.132142+0000","last_clean":"2023-10-30T12:45:19.132142+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:19.132142+0000","last_undegraded":"2023-10-30T12:45:19.132142+0000","last_fullsized":"2023-10-30T12:45:19.132142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:31.748472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067448+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:19.067448+0000","last_peered":"2023-10-30T12:45:19.067448+0000","last_clean":"2023-10-30T12:45:19.067448+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:19.067448+0000","last_undegraded":"2023-10-30T12:45:19.067448+0000","last_fullsized":"2023-10-30T12:45:19.067448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125571+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:19.125571+0000","last_peered":"2023-10-30T12:45:19.125571+0000","last_clean":"2023-10-30T12:45:19.125571+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:19.125571+0000","last_undegraded":"2023-10-30T12:45:19.125571+0000","last_fullsized":"2023-10-30T12:45:19.125571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:37:15.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074024+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:19.074024+0000","last_peered":"2023-10-30T12:45:19.074024+0000","last_clean":"2023-10-30T12:45:19.074024+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:19.074024+0000","last_undegraded":"2023-10-30T12:45:19.074024+0000","last_fullsized":"2023-10-30T12:45:19.074024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125593+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:19.125593+0000","last_peered":"2023-10-30T12:45:19.125593+0000","last_clean":"2023-10-30T12:45:19.125593+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:19.125593+0000","last_undegraded":"2023-10-30T12:45:19.125593+0000","last_fullsized":"2023-10-30T12:45:19.125593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:59:22.510041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.090865+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:19.090865+0000","last_peered":"2023-10-30T12:45:19.090865+0000","last_clean":"2023-10-30T12:45:19.090865+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:19.090865+0000","last_undegraded":"2023-10-30T12:45:19.090865+0000","last_fullsized":"2023-10-30T12:45:19.090865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123170+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:19.123170+0000","last_peered":"2023-10-30T12:45:19.123170+0000","last_clean":"2023-10-30T12:45:19.123170+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:19.123170+0000","last_undegraded":"2023-10-30T12:45:19.123170+0000","last_fullsized":"2023-10-30T12:45:19.123170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:49:48.282737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.103701+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:19.103701+0000","last_peered":"2023-10-30T12:45:19.103701+0000","last_clean":"2023-10-30T12:45:19.103701+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:19.103701+0000","last_undegraded":"2023-10-30T12:45:19.103701+0000","last_fullsized":"2023-10-30T12:45:19.103701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131007+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:19.131007+0000","last_peered":"2023-10-30T12:45:19.131007+0000","last_clean":"2023-10-30T12:45:19.131007+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:19.131007+0000","last_undegraded":"2023-10-30T12:45:19.131007+0000","last_fullsized":"2023-10-30T12:45:19.131007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:37:14.750419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071237+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:19.071237+0000","last_peered":"2023-10-30T12:45:19.071237+0000","last_clean":"2023-10-30T12:45:19.071237+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:19.071237+0000","last_undegraded":"2023-10-30T12:45:19.071237+0000","last_fullsized":"2023-10-30T12:45:19.071237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100963+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:19.100963+0000","last_peered":"2023-10-30T12:45:19.100963+0000","last_clean":"2023-10-30T12:45:19.100963+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:19.100963+0000","last_undegraded":"2023-10-30T12:45:19.100963+0000","last_fullsized":"2023-10-30T12:45:19.100963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:24:07.009328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.073756+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:19.073756+0000","last_peered":"2023-10-30T12:45:19.073756+0000","last_clean":"2023-10-30T12:45:19.073756+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:19.073756+0000","last_undegraded":"2023-10-30T12:45:19.073756+0000","last_fullsized":"2023-10-30T12:45:19.073756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125429+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:19.125429+0000","last_peered":"2023-10-30T12:45:19.125429+0000","last_clean":"2023-10-30T12:45:19.125429+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:19.125429+0000","last_undegraded":"2023-10-30T12:45:19.125429+0000","last_fullsized":"2023-10-30T12:45:19.125429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:17:21.758467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070370+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:19.070370+0000","last_peered":"2023-10-30T12:45:19.070370+0000","last_clean":"2023-10-30T12:45:19.070370+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:19.070370+0000","last_undegraded":"2023-10-30T12:45:19.070370+0000","last_fullsized":"2023-10-30T12:45:19.070370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125233+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:19.125233+0000","last_peered":"2023-10-30T12:45:19.125233+0000","last_clean":"2023-10-30T12:45:19.125233+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:19.125233+0000","last_undegraded":"2023-10-30T12:45:19.125233+0000","last_fullsized":"2023-10-30T12:45:19.125233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:10:00.599467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074783+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:19.074783+0000","last_peered":"2023-10-30T12:45:19.074783+0000","last_clean":"2023-10-30T12:45:19.074783+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:19.074783+0000","last_undegraded":"2023-10-30T12:45:19.074783+0000","last_fullsized":"2023-10-30T12:45:19.074783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125703+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:19.125703+0000","last_peered":"2023-10-30T12:45:19.125703+0000","last_clean":"2023-10-30T12:45:19.125703+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:19.125703+0000","last_undegraded":"2023-10-30T12:45:19.125703+0000","last_fullsized":"2023-10-30T12:45:19.125703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:08.273130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067586+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:19.067586+0000","last_peered":"2023-10-30T12:45:19.067586+0000","last_clean":"2023-10-30T12:45:19.067586+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:19.067586+0000","last_undegraded":"2023-10-30T12:45:19.067586+0000","last_fullsized":"2023-10-30T12:45:19.067586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134283+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:19.134283+0000","last_peered":"2023-10-30T12:45:19.134283+0000","last_clean":"2023-10-30T12:45:19.134283+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:19.134283+0000","last_undegraded":"2023-10-30T12:45:19.134283+0000","last_fullsized":"2023-10-30T12:45:19.134283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:21.306649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.075237+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:19.075237+0000","last_peered":"2023-10-30T12:45:19.075237+0000","last_clean":"2023-10-30T12:45:19.075237+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:19.075237+0000","last_undegraded":"2023-10-30T12:45:19.075237+0000","last_fullsized":"2023-10-30T12:45:19.075237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131379+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:19.131379+0000","last_peered":"2023-10-30T12:45:19.131379+0000","last_clean":"2023-10-30T12:45:19.131379+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:19.131379+0000","last_undegraded":"2023-10-30T12:45:19.131379+0000","last_fullsized":"2023-10-30T12:45:19.131379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:19.213457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-30T12:45:18.103394+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:18.103394+0000","last_peered":"2023-10-30T12:45:18.103394+0000","last_clean":"2023-10-30T12:45:18.103394+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:18.103394+0000","last_undegraded":"2023-10-30T12:45:18.103394+0000","last_fullsized":"2023-10-30T12:45:18.103394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.087815+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:19.087815+0000","last_peered":"2023-10-30T12:45:19.087815+0000","last_clean":"2023-10-30T12:45:19.087815+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:19.087815+0000","last_undegraded":"2023-10-30T12:45:19.087815+0000","last_fullsized":"2023-10-30T12:45:19.087815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125763+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:19.125763+0000","last_peered":"2023-10-30T12:45:19.125763+0000","last_clean":"2023-10-30T12:45:19.125763+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:19.125763+0000","last_undegraded":"2023-10-30T12:45:19.125763+0000","last_fullsized":"2023-10-30T12:45:19.125763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:12.458726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070897+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:19.070897+0000","last_peered":"2023-10-30T12:45:19.070897+0000","last_clean":"2023-10-30T12:45:19.070897+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:19.070897+0000","last_undegraded":"2023-10-30T12:45:19.070897+0000","last_fullsized":"2023-10-30T12:45:19.070897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123233+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:19.123233+0000","last_peered":"2023-10-30T12:45:19.123233+0000","last_clean":"2023-10-30T12:45:19.123233+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:19.123233+0000","last_undegraded":"2023-10-30T12:45:19.123233+0000","last_fullsized":"2023-10-30T12:45:19.123233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:45:47.965390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074588+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:19.074588+0000","last_peered":"2023-10-30T12:45:19.074588+0000","last_clean":"2023-10-30T12:45:19.074588+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:19.074588+0000","last_undegraded":"2023-10-30T12:45:19.074588+0000","last_fullsized":"2023-10-30T12:45:19.074588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.076603+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:19.076603+0000","last_peered":"2023-10-30T12:45:19.076603+0000","last_clean":"2023-10-30T12:45:19.076603+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:19.076603+0000","last_undegraded":"2023-10-30T12:45:19.076603+0000","last_fullsized":"2023-10-30T12:45:19.076603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:18.509502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.104036+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:19.104036+0000","last_peered":"2023-10-30T12:45:19.104036+0000","last_clean":"2023-10-30T12:45:19.104036+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:19.104036+0000","last_undegraded":"2023-10-30T12:45:19.104036+0000","last_fullsized":"2023-10-30T12:45:19.104036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129905+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:19.129905+0000","last_peered":"2023-10-30T12:45:19.129905+0000","last_clean":"2023-10-30T12:45:19.129905+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:19.129905+0000","last_undegraded":"2023-10-30T12:45:19.129905+0000","last_fullsized":"2023-10-30T12:45:19.129905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:27.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066874+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.066874+0000","last_peered":"2023-10-30T12:45:19.066874+0000","last_clean":"2023-10-30T12:45:19.066874+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.066874+0000","last_undegraded":"2023-10-30T12:45:19.066874+0000","last_fullsized":"2023-10-30T12:45:19.066874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131160+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:19.131160+0000","last_peered":"2023-10-30T12:45:19.131160+0000","last_clean":"2023-10-30T12:45:19.131160+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:19.131160+0000","last_undegraded":"2023-10-30T12:45:19.131160+0000","last_fullsized":"2023-10-30T12:45:19.131160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:29:23.885208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.079461+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:19.079461+0000","last_peered":"2023-10-30T12:45:19.079461+0000","last_clean":"2023-10-30T12:45:19.079461+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:19.079461+0000","last_undegraded":"2023-10-30T12:45:19.079461+0000","last_fullsized":"2023-10-30T12:45:19.079461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.081681+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:19.081681+0000","last_peered":"2023-10-30T12:45:19.081681+0000","last_clean":"2023-10-30T12:45:19.081681+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:19.081681+0000","last_undegraded":"2023-10-30T12:45:19.081681+0000","last_fullsized":"2023-10-30T12:45:19.081681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:43:34.496700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066804+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:19.066804+0000","last_peered":"2023-10-30T12:45:19.066804+0000","last_clean":"2023-10-30T12:45:19.066804+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:19.066804+0000","last_undegraded":"2023-10-30T12:45:19.066804+0000","last_fullsized":"2023-10-30T12:45:19.066804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.113494+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:19.113494+0000","last_peered":"2023-10-30T12:45:19.113494+0000","last_clean":"2023-10-30T12:45:19.113494+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:19.113494+0000","last_undegraded":"2023-10-30T12:45:19.113494+0000","last_fullsized":"2023-10-30T12:45:19.113494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:55.006512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074930+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:19.074930+0000","last_peered":"2023-10-30T12:45:19.074930+0000","last_clean":"2023-10-30T12:45:19.074930+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:19.074930+0000","last_undegraded":"2023-10-30T12:45:19.074930+0000","last_fullsized":"2023-10-30T12:45:19.074930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097149+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:19.097149+0000","last_peered":"2023-10-30T12:45:19.097149+0000","last_clean":"2023-10-30T12:45:19.097149+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:19.097149+0000","last_undegraded":"2023-10-30T12:45:19.097149+0000","last_fullsized":"2023-10-30T12:45:19.097149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:50:30.460567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070532+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:19.070532+0000","last_peered":"2023-10-30T12:45:19.070532+0000","last_clean":"2023-10-30T12:45:19.070532+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:19.070532+0000","last_undegraded":"2023-10-30T12:45:19.070532+0000","last_fullsized":"2023-10-30T12:45:19.070532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.117315+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:19.117315+0000","last_peered":"2023-10-30T12:45:19.117315+0000","last_clean":"2023-10-30T12:45:19.117315+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:19.117315+0000","last_undegraded":"2023-10-30T12:45:19.117315+0000","last_fullsized":"2023-10-30T12:45:19.117315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:50.870760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067994+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:19.067994+0000","last_peered":"2023-10-30T12:45:19.067994+0000","last_clean":"2023-10-30T12:45:19.067994+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:19.067994+0000","last_undegraded":"2023-10-30T12:45:19.067994+0000","last_fullsized":"2023-10-30T12:45:19.067994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.078936+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:19.078936+0000","last_peered":"2023-10-30T12:45:19.078936+0000","last_clean":"2023-10-30T12:45:19.078936+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:19.078936+0000","last_undegraded":"2023-10-30T12:45:19.078936+0000","last_fullsized":"2023-10-30T12:45:19.078936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:27.368058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066164+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:19.066164+0000","last_peered":"2023-10-30T12:45:19.066164+0000","last_clean":"2023-10-30T12:45:19.066164+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:19.066164+0000","last_undegraded":"2023-10-30T12:45:19.066164+0000","last_fullsized":"2023-10-30T12:45:19.066164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099909+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:19.099909+0000","last_peered":"2023-10-30T12:45:19.099909+0000","last_clean":"2023-10-30T12:45:19.099909+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:19.099909+0000","last_undegraded":"2023-10-30T12:45:19.099909+0000","last_fullsized":"2023-10-30T12:45:19.099909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:38:28.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.081123+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:19.081123+0000","last_peered":"2023-10-30T12:45:19.081123+0000","last_clean":"2023-10-30T12:45:19.081123+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:19.081123+0000","last_undegraded":"2023-10-30T12:45:19.081123+0000","last_fullsized":"2023-10-30T12:45:19.081123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.102712+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:19.102712+0000","last_peered":"2023-10-30T12:45:19.102712+0000","last_clean":"2023-10-30T12:45:19.102712+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:19.102712+0000","last_undegraded":"2023-10-30T12:45:19.102712+0000","last_fullsized":"2023-10-30T12:45:19.102712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:53:50.701760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.135999+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:19.135999+0000","last_peered":"2023-10-30T12:45:19.135999+0000","last_clean":"2023-10-30T12:45:19.135999+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:19.135999+0000","last_undegraded":"2023-10-30T12:45:19.135999+0000","last_fullsized":"2023-10-30T12:45:19.135999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:51:55.133787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071737+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:19.071737+0000","last_peered":"2023-10-30T12:45:19.071737+0000","last_clean":"2023-10-30T12:45:19.071737+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:19.071737+0000","last_undegraded":"2023-10-30T12:45:19.071737+0000","last_fullsized":"2023-10-30T12:45:19.071737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.135971+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:19.135971+0000","last_peered":"2023-10-30T12:45:19.135971+0000","last_clean":"2023-10-30T12:45:19.135971+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:19.135971+0000","last_undegraded":"2023-10-30T12:45:19.135971+0000","last_fullsized":"2023-10-30T12:45:19.135971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:27.039964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.079089+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:19.079089+0000","last_peered":"2023-10-30T12:45:19.079089+0000","last_clean":"2023-10-30T12:45:19.079089+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:19.079089+0000","last_undegraded":"2023-10-30T12:45:19.079089+0000","last_fullsized":"2023-10-30T12:45:19.079089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.132171+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:19.132171+0000","last_peered":"2023-10-30T12:45:19.132171+0000","last_clean":"2023-10-30T12:45:19.132171+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:19.132171+0000","last_undegraded":"2023-10-30T12:45:19.132171+0000","last_fullsized":"2023-10-30T12:45:19.132171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:16.015616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.072170+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:19.072170+0000","last_peered":"2023-10-30T12:45:19.072170+0000","last_clean":"2023-10-30T12:45:19.072170+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:19.072170+0000","last_undegraded":"2023-10-30T12:45:19.072170+0000","last_fullsized":"2023-10-30T12:45:19.072170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129361+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:19.129361+0000","last_peered":"2023-10-30T12:45:19.129361+0000","last_clean":"2023-10-30T12:45:19.129361+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:19.129361+0000","last_undegraded":"2023-10-30T12:45:19.129361+0000","last_fullsized":"2023-10-30T12:45:19.129361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:35:54.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.089060+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:19.089060+0000","last_peered":"2023-10-30T12:45:19.089060+0000","last_clean":"2023-10-30T12:45:19.089060+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:19.089060+0000","last_undegraded":"2023-10-30T12:45:19.089060+0000","last_fullsized":"2023-10-30T12:45:19.089060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":648191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":653947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:25.227 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:25.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.401+0000 7f97ce794640 1 Processor -- start 2023-10-30T12:45:25.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.401+0000 7f97ce794640 1 -- start start 2023-10-30T12:45:25.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ce794640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f97c806bb40 0x7f97c8157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ce794640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97c806c420 0x7f97c8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ce794640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c8071e00 con 0x7f97c806b760 2023-10-30T12:45:25.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ce794640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c8070020 con 0x7f97c806bb40 2023-10-30T12:45:25.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ce794640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c8070f10 con 0x7f97c806c420 2023-10-30T12:45:25.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c7fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f97c806b760 0x7f97c815e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44736/0 (socket says 172.21.15.8:44736) 2023-10-30T12:45:25.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c7fff640 1 -- 172.21.15.8:0/1428327590 learned_addr learned my addr 172.21.15.8:0/1428327590 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:25.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c77fe640 1 --2- 172.21.15.8:0/1428327590 >> v2:172.21.15.8:3300/0 conn(0x7f97c806bb40 0x7f97c8157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97ccd31640 1 --2- 172.21.15.8:0/1428327590 >> v2:172.21.15.46:3300/0 conn(0x7f97c806c420 0x7f97c8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c6ffd640 1 -- 172.21.15.8:0/1428327590 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214134780 0 0) 0x7f97c8071e00 con 0x7f97c806b760 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c6ffd640 1 -- 172.21.15.8:0/1428327590 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b0003530 con 0x7f97c806b760 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c77fe640 1 -- 172.21.15.8:0/1428327590 >> v2:172.21.15.46:3300/0 conn(0x7f97c806c420 msgr2=0x7f97c8157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c77fe640 1 --2- 172.21.15.8:0/1428327590 >> v2:172.21.15.46:3300/0 conn(0x7f97c806c420 0x7f97c8157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c77fe640 1 -- 172.21.15.8:0/1428327590 >> v1:172.21.15.8:6789/0 conn(0x7f97c806b760 legacy=0x7f97c815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.405+0000 7f97c77fe640 1 -- 172.21.15.8:0/1428327590 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80711f0 con 0x7f97c806bb40 2023-10-30T12:45:25.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97c77fe640 1 --2- 172.21.15.8:0/1428327590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8157a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97c815b7a0 tx=0x7f97bc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114bdcce51ebd296 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97c6ffd640 1 -- 172.21.15.8:0/1428327590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0a7070 con 0x7f97c806bb40 2023-10-30T12:45:25.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97c6ffd640 1 -- 172.21.15.8:0/1428327590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc0ab430 con 0x7f97c806bb40 2023-10-30T12:45:25.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97c6ffd640 1 -- 172.21.15.8:0/1428327590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0ab7c0 con 0x7f97c806bb40 2023-10-30T12:45:25.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- 172.21.15.8:0/1428327590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 msgr2=0x7f97c8157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- 172.21.15.8:0/1428327590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8157a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97c815b7a0 tx=0x7f97bc01c430 comp rx=0 tx=0).stop 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- 172.21.15.8:0/1428327590 shutdown_connections 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- 172.21.15.8:0/1428327590 >> v2:172.21.15.46:3300/0 conn(0x7f97c806c420 0x7f97c8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- 172.21.15.8:0/1428327590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8157a20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- 172.21.15.8:0/1428327590 >> 172.21.15.8:0/1428327590 conn(0x7f97c80cac70 msgr2=0x7f97c80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:25.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- 172.21.15.8:0/1428327590 shutdown_connections 2023-10-30T12:45:25.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- 172.21.15.8:0/1428327590 wait complete. 2023-10-30T12:45:25.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 Processor -- start 2023-10-30T12:45:25.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- start start 2023-10-30T12:45:25.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97c806b760 0x7f97c815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c806c420 0x7f97c81f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c814b7f0 con 0x7f97c806bb40 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c9500 con 0x7f97c806c420 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.409+0000 7f97ce794640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c814ad80 con 0x7f97c806b760 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36092/0 (socket says 172.21.15.8:36092) 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 -- 172.21.15.8:0/565928111 learned_addr learned my addr 172.21.15.8:0/565928111 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97ccd31640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c806c420 0x7f97c81f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c7fff640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97c806b760 0x7f97c815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 -- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97c806b760 msgr2=0x7f97c815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97c806b760 0x7f97c815ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 -- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c806c420 msgr2=0x7f97c81f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c806c420 0x7f97c81f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 -- 172.21.15.8:0/565928111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc009420 con 0x7f97c806bb40 2023-10-30T12:45:25.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c77fe640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ad040 tx=0x7f97bc01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0abe50 con 0x7f97c806bb40 2023-10-30T12:45:25.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc0bf410 con 0x7f97c806bb40 2023-10-30T12:45:25.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80c8eb0 con 0x7f97c806bb40 2023-10-30T12:45:25.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0bf7c0 con 0x7f97c806bb40 2023-10-30T12:45:25.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80c9990 con 0x7f97c806bb40 2023-10-30T12:45:25.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c804e170 con 0x7f97c806bb40 2023-10-30T12:45:25.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.413+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97bc0bf920 con 0x7f97c806bb40 2023-10-30T12:45:25.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.417+0000 7f97c4ff9640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 0x7f97b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.417+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97bc11e610 con 0x7f97c806bb40 2023-10-30T12:45:25.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.417+0000 7f97c7fff640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 0x7f97b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.417+0000 7f97c7fff640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 0x7f97b4068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97b80011f0 tx=0x7f97b8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.421+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97bc0ebb20 con 0x7f97c806bb40 2023-10-30T12:45:25.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.573+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97c8070c10 con 0x7f97b40664a0 2023-10-30T12:45:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.581+0000 7f97c4ff9640 1 -- 172.21.15.8:0/565928111 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264409 (secure 0 0 0) 0x7f97b8001cf0 con 0x7f97b40664a0 2023-10-30T12:45:25.587 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:25.588 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:25.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 msgr2=0x7f97b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 0x7f97b4068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97b80011f0 tx=0x7f97b8009040 comp rx=0 tx=0).stop 2023-10-30T12:45:25.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 msgr2=0x7f97c8151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ad040 tx=0x7f97bc01ac70 comp rx=0 tx=0).stop 2023-10-30T12:45:25.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 shutdown_connections 2023-10-30T12:45:25.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97b40664a0 0x7f97b4068960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c806c420 0x7f97c81f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97c806bb40 0x7f97c8151bb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 --2- 172.21.15.8:0/565928111 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97c806b760 0x7f97c815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 >> 172.21.15.8:0/565928111 conn(0x7f97c80cac70 msgr2=0x7f97c80733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 shutdown_connections 2023-10-30T12:45:25.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.585+0000 7f97ce794640 1 -- 172.21.15.8:0/565928111 wait complete. 2023-10-30T12:45:25.607 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-30T12:45:23.609604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252068,"kb_used_data":2612,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722652,"statfs":{"total":773094113280,"available":772835995648,"internally_reserved":0,"allocated":2674688,"data_stored":1669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001165"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125671+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:19.125671+0000","last_peered":"2023-10-30T12:45:19.125671+0000","last_clean":"2023-10-30T12:45:19.125671+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:19.125671+0000","last_undegraded":"2023-10-30T12:45:19.125671+0000","last_fullsized":"2023-10-30T12:45:19.125671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:09.263692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.087969+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:19.087969+0000","last_peered":"2023-10-30T12:45:19.087969+0000","last_clean":"2023-10-30T12:45:19.087969+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:19.087969+0000","last_undegraded":"2023-10-30T12:45:19.087969+0000","last_fullsized":"2023-10-30T12:45:19.087969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:01:17.337653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100587+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:19.100587+0000","last_peered":"2023-10-30T12:45:19.100587+0000","last_clean":"2023-10-30T12:45:19.100587+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:19.100587+0000","last_undegraded":"2023-10-30T12:45:19.100587+0000","last_fullsized":"2023-10-30T12:45:19.100587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:15:42.050956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134450+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:19.134450+0000","last_peered":"2023-10-30T12:45:19.134450+0000","last_clean":"2023-10-30T12:45:19.134450+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:19.134450+0000","last_undegraded":"2023-10-30T12:45:19.134450+0000","last_fullsized":"2023-10-30T12:45:19.134450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:02:49.883736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125662+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:19.125662+0000","last_peered":"2023-10-30T12:45:19.125662+0000","last_clean":"2023-10-30T12:45:19.125662+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:19.125662+0000","last_undegraded":"2023-10-30T12:45:19.125662+0000","last_fullsized":"2023-10-30T12:45:19.125662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.623327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130904+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:19.130904+0000","last_peered":"2023-10-30T12:45:19.130904+0000","last_clean":"2023-10-30T12:45:19.130904+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:19.130904+0000","last_undegraded":"2023-10-30T12:45:19.130904+0000","last_fullsized":"2023-10-30T12:45:19.130904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:30:48.258797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125473+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:19.125473+0000","last_peered":"2023-10-30T12:45:19.125473+0000","last_clean":"2023-10-30T12:45:19.125473+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:19.125473+0000","last_undegraded":"2023-10-30T12:45:19.125473+0000","last_fullsized":"2023-10-30T12:45:19.125473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:04:10.662268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.122905+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:19.122905+0000","last_peered":"2023-10-30T12:45:19.122905+0000","last_clean":"2023-10-30T12:45:19.122905+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:19.122905+0000","last_undegraded":"2023-10-30T12:45:19.122905+0000","last_fullsized":"2023-10-30T12:45:19.122905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:57.965860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.136565+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:19.136565+0000","last_peered":"2023-10-30T12:45:19.136565+0000","last_clean":"2023-10-30T12:45:19.136565+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:19.136565+0000","last_undegraded":"2023-10-30T12:45:19.136565+0000","last_fullsized":"2023-10-30T12:45:19.136565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:36:07.602135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130992+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:19.130992+0000","last_peered":"2023-10-30T12:45:19.130992+0000","last_clean":"2023-10-30T12:45:19.130992+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:19.130992+0000","last_undegraded":"2023-10-30T12:45:19.130992+0000","last_fullsized":"2023-10-30T12:45:19.130992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:11:52.175612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.130935+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:19.130935+0000","last_peered":"2023-10-30T12:45:19.130935+0000","last_clean":"2023-10-30T12:45:19.130935+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:19.130935+0000","last_undegraded":"2023-10-30T12:45:19.130935+0000","last_fullsized":"2023-10-30T12:45:19.130935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:57.478243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.120288+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:19.120288+0000","last_peered":"2023-10-30T12:45:19.120288+0000","last_clean":"2023-10-30T12:45:19.120288+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:19.120288+0000","last_undegraded":"2023-10-30T12:45:19.120288+0000","last_fullsized":"2023-10-30T12:45:19.120288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:08.246156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129373+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:19.129373+0000","last_peered":"2023-10-30T12:45:19.129373+0000","last_clean":"2023-10-30T12:45:19.129373+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:19.129373+0000","last_undegraded":"2023-10-30T12:45:19.129373+0000","last_fullsized":"2023-10-30T12:45:19.129373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:47.272376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.098658+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:19.098658+0000","last_peered":"2023-10-30T12:45:19.098658+0000","last_clean":"2023-10-30T12:45:19.098658+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:19.098658+0000","last_undegraded":"2023-10-30T12:45:19.098658+0000","last_fullsized":"2023-10-30T12:45:19.098658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:51:02.515760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.095835+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:19.095835+0000","last_peered":"2023-10-30T12:45:19.095835+0000","last_clean":"2023-10-30T12:45:19.095835+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:19.095835+0000","last_undegraded":"2023-10-30T12:45:19.095835+0000","last_fullsized":"2023-10-30T12:45:19.095835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:29:31.115101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.118982+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:19.118982+0000","last_peered":"2023-10-30T12:45:19.118982+0000","last_clean":"2023-10-30T12:45:19.118982+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:19.118982+0000","last_undegraded":"2023-10-30T12:45:19.118982+0000","last_fullsized":"2023-10-30T12:45:19.118982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:26:59.349423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134339+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:19.134339+0000","last_peered":"2023-10-30T12:45:19.134339+0000","last_clean":"2023-10-30T12:45:19.134339+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:19.134339+0000","last_undegraded":"2023-10-30T12:45:19.134339+0000","last_fullsized":"2023-10-30T12:45:19.134339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:23:45.435309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097633+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:19.097633+0000","last_peered":"2023-10-30T12:45:19.097633+0000","last_clean":"2023-10-30T12:45:19.097633+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:19.097633+0000","last_undegraded":"2023-10-30T12:45:19.097633+0000","last_fullsized":"2023-10-30T12:45:19.097633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:38.269819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125544+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:19.125544+0000","last_peered":"2023-10-30T12:45:19.125544+0000","last_clean":"2023-10-30T12:45:19.125544+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:19.125544+0000","last_undegraded":"2023-10-30T12:45:19.125544+0000","last_fullsized":"2023-10-30T12:45:19.125544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:16:32.267045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.102834+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:19.102834+0000","last_peered":"2023-10-30T12:45:19.102834+0000","last_clean":"2023-10-30T12:45:19.102834+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:19.102834+0000","last_undegraded":"2023-10-30T12:45:19.102834+0000","last_fullsized":"2023-10-30T12:45:19.102834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:04:53.521179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.122104+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:19.122104+0000","last_peered":"2023-10-30T12:45:19.122104+0000","last_clean":"2023-10-30T12:45:19.122104+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:19.122104+0000","last_undegraded":"2023-10-30T12:45:19.122104+0000","last_fullsized":"2023-10-30T12:45:19.122104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:24:55.744168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134314+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:19.134314+0000","last_peered":"2023-10-30T12:45:19.134314+0000","last_clean":"2023-10-30T12:45:19.134314+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:19.134314+0000","last_undegraded":"2023-10-30T12:45:19.134314+0000","last_fullsized":"2023-10-30T12:45:19.134314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:41:59.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099728+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:19.099728+0000","last_peered":"2023-10-30T12:45:19.099728+0000","last_clean":"2023-10-30T12:45:19.099728+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:19.099728+0000","last_undegraded":"2023-10-30T12:45:19.099728+0000","last_fullsized":"2023-10-30T12:45:19.099728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:41:04.653693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.118908+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:19.118908+0000","last_peered":"2023-10-30T12:45:19.118908+0000","last_clean":"2023-10-30T12:45:19.118908+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:19.118908+0000","last_undegraded":"2023-10-30T12:45:19.118908+0000","last_fullsized":"2023-10-30T12:45:19.118908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:24:03.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100574+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:19.100574+0000","last_peered":"2023-10-30T12:45:19.100574+0000","last_clean":"2023-10-30T12:45:19.100574+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:19.100574+0000","last_undegraded":"2023-10-30T12:45:19.100574+0000","last_fullsized":"2023-10-30T12:45:19.100574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:06:57.478243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.115894+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:19.115894+0000","last_peered":"2023-10-30T12:45:19.115894+0000","last_clean":"2023-10-30T12:45:19.115894+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:19.115894+0000","last_undegraded":"2023-10-30T12:45:19.115894+0000","last_fullsized":"2023-10-30T12:45:19.115894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:38:28.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.085908+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:19.085908+0000","last_peered":"2023-10-30T12:45:19.085908+0000","last_clean":"2023-10-30T12:45:19.085908+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:19.085908+0000","last_undegraded":"2023-10-30T12:45:19.085908+0000","last_fullsized":"2023-10-30T12:45:19.085908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:46.957164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099767+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:19.099767+0000","last_peered":"2023-10-30T12:45:19.099767+0000","last_clean":"2023-10-30T12:45:19.099767+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:19.099767+0000","last_undegraded":"2023-10-30T12:45:19.099767+0000","last_fullsized":"2023-10-30T12:45:19.099767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:36.269949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097184+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:19.097184+0000","last_peered":"2023-10-30T12:45:19.097184+0000","last_clean":"2023-10-30T12:45:19.097184+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:19.097184+0000","last_undegraded":"2023-10-30T12:45:19.097184+0000","last_fullsized":"2023-10-30T12:45:19.097184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:05:47.748667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.119876+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:19.119876+0000","last_peered":"2023-10-30T12:45:19.119876+0000","last_clean":"2023-10-30T12:45:19.119876+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:19.119876+0000","last_undegraded":"2023-10-30T12:45:19.119876+0000","last_fullsized":"2023-10-30T12:45:19.119876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:14:05.237539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.108663+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:19.108663+0000","last_peered":"2023-10-30T12:45:19.108663+0000","last_clean":"2023-10-30T12:45:19.108663+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:19.108663+0000","last_undegraded":"2023-10-30T12:45:19.108663+0000","last_fullsized":"2023-10-30T12:45:19.108663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:02:22.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.101017+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:19.101017+0000","last_peered":"2023-10-30T12:45:19.101017+0000","last_clean":"2023-10-30T12:45:19.101017+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:19.101017+0000","last_undegraded":"2023-10-30T12:45:19.101017+0000","last_fullsized":"2023-10-30T12:45:19.101017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:37:54.417550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.117268+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:19.117268+0000","last_peered":"2023-10-30T12:45:19.117268+0000","last_clean":"2023-10-30T12:45:19.117268+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:19.117268+0000","last_undegraded":"2023-10-30T12:45:19.117268+0000","last_fullsized":"2023-10-30T12:45:19.117268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:43:34.496700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074000+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:19.074000+0000","last_peered":"2023-10-30T12:45:19.074000+0000","last_clean":"2023-10-30T12:45:19.074000+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:19.074000+0000","last_undegraded":"2023-10-30T12:45:19.074000+0000","last_fullsized":"2023-10-30T12:45:19.074000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123644+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:19.123644+0000","last_peered":"2023-10-30T12:45:19.123644+0000","last_clean":"2023-10-30T12:45:19.123644+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:19.123644+0000","last_undegraded":"2023-10-30T12:45:19.123644+0000","last_fullsized":"2023-10-30T12:45:19.123644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:04:59.171051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071102+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:19.071102+0000","last_peered":"2023-10-30T12:45:19.071102+0000","last_clean":"2023-10-30T12:45:19.071102+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:19.071102+0000","last_undegraded":"2023-10-30T12:45:19.071102+0000","last_fullsized":"2023-10-30T12:45:19.071102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.072044+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:19.072044+0000","last_peered":"2023-10-30T12:45:19.072044+0000","last_clean":"2023-10-30T12:45:19.072044+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:19.072044+0000","last_undegraded":"2023-10-30T12:45:19.072044+0000","last_fullsized":"2023-10-30T12:45:19.072044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.090343+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:19.090343+0000","last_peered":"2023-10-30T12:45:19.090343+0000","last_clean":"2023-10-30T12:45:19.090343+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:19.090343+0000","last_undegraded":"2023-10-30T12:45:19.090343+0000","last_fullsized":"2023-10-30T12:45:19.090343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:55:45.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.106131+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:19.106131+0000","last_peered":"2023-10-30T12:45:19.106131+0000","last_clean":"2023-10-30T12:45:19.106131+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:19.106131+0000","last_undegraded":"2023-10-30T12:45:19.106131+0000","last_fullsized":"2023-10-30T12:45:19.106131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.085014+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:19.085014+0000","last_peered":"2023-10-30T12:45:19.085014+0000","last_clean":"2023-10-30T12:45:19.085014+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:19.085014+0000","last_undegraded":"2023-10-30T12:45:19.085014+0000","last_fullsized":"2023-10-30T12:45:19.085014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:50.870760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071123+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:19.071123+0000","last_peered":"2023-10-30T12:45:19.071123+0000","last_clean":"2023-10-30T12:45:19.071123+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:19.071123+0000","last_undegraded":"2023-10-30T12:45:19.071123+0000","last_fullsized":"2023-10-30T12:45:19.071123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134275+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:19.134275+0000","last_peered":"2023-10-30T12:45:19.134275+0000","last_clean":"2023-10-30T12:45:19.134275+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:19.134275+0000","last_undegraded":"2023-10-30T12:45:19.134275+0000","last_fullsized":"2023-10-30T12:45:19.134275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:24:47.493636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.113411+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:19.113411+0000","last_peered":"2023-10-30T12:45:19.113411+0000","last_clean":"2023-10-30T12:45:19.113411+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:19.113411+0000","last_undegraded":"2023-10-30T12:45:19.113411+0000","last_fullsized":"2023-10-30T12:45:19.113411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.132142+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:19.132142+0000","last_peered":"2023-10-30T12:45:19.132142+0000","last_clean":"2023-10-30T12:45:19.132142+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:19.132142+0000","last_undegraded":"2023-10-30T12:45:19.132142+0000","last_fullsized":"2023-10-30T12:45:19.132142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:49:31.748472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067448+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:19.067448+0000","last_peered":"2023-10-30T12:45:19.067448+0000","last_clean":"2023-10-30T12:45:19.067448+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:19.067448+0000","last_undegraded":"2023-10-30T12:45:19.067448+0000","last_fullsized":"2023-10-30T12:45:19.067448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125571+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:19.125571+0000","last_peered":"2023-10-30T12:45:19.125571+0000","last_clean":"2023-10-30T12:45:19.125571+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:19.125571+0000","last_undegraded":"2023-10-30T12:45:19.125571+0000","last_fullsized":"2023-10-30T12:45:19.125571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:37:15.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074024+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:19.074024+0000","last_peered":"2023-10-30T12:45:19.074024+0000","last_clean":"2023-10-30T12:45:19.074024+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:19.074024+0000","last_undegraded":"2023-10-30T12:45:19.074024+0000","last_fullsized":"2023-10-30T12:45:19.074024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125593+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:19.125593+0000","last_peered":"2023-10-30T12:45:19.125593+0000","last_clean":"2023-10-30T12:45:19.125593+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:19.125593+0000","last_undegraded":"2023-10-30T12:45:19.125593+0000","last_fullsized":"2023-10-30T12:45:19.125593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:59:22.510041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.090865+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:19.090865+0000","last_peered":"2023-10-30T12:45:19.090865+0000","last_clean":"2023-10-30T12:45:19.090865+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:19.090865+0000","last_undegraded":"2023-10-30T12:45:19.090865+0000","last_fullsized":"2023-10-30T12:45:19.090865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123170+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:19.123170+0000","last_peered":"2023-10-30T12:45:19.123170+0000","last_clean":"2023-10-30T12:45:19.123170+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:19.123170+0000","last_undegraded":"2023-10-30T12:45:19.123170+0000","last_fullsized":"2023-10-30T12:45:19.123170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:49:48.282737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.103701+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:19.103701+0000","last_peered":"2023-10-30T12:45:19.103701+0000","last_clean":"2023-10-30T12:45:19.103701+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:19.103701+0000","last_undegraded":"2023-10-30T12:45:19.103701+0000","last_fullsized":"2023-10-30T12:45:19.103701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131007+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:19.131007+0000","last_peered":"2023-10-30T12:45:19.131007+0000","last_clean":"2023-10-30T12:45:19.131007+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:19.131007+0000","last_undegraded":"2023-10-30T12:45:19.131007+0000","last_fullsized":"2023-10-30T12:45:19.131007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:37:14.750419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071237+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:19.071237+0000","last_peered":"2023-10-30T12:45:19.071237+0000","last_clean":"2023-10-30T12:45:19.071237+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:19.071237+0000","last_undegraded":"2023-10-30T12:45:19.071237+0000","last_fullsized":"2023-10-30T12:45:19.071237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.100963+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:19.100963+0000","last_peered":"2023-10-30T12:45:19.100963+0000","last_clean":"2023-10-30T12:45:19.100963+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:19.100963+0000","last_undegraded":"2023-10-30T12:45:19.100963+0000","last_fullsized":"2023-10-30T12:45:19.100963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:24:07.009328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.073756+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:19.073756+0000","last_peered":"2023-10-30T12:45:19.073756+0000","last_clean":"2023-10-30T12:45:19.073756+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:19.073756+0000","last_undegraded":"2023-10-30T12:45:19.073756+0000","last_fullsized":"2023-10-30T12:45:19.073756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125429+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:19.125429+0000","last_peered":"2023-10-30T12:45:19.125429+0000","last_clean":"2023-10-30T12:45:19.125429+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:19.125429+0000","last_undegraded":"2023-10-30T12:45:19.125429+0000","last_fullsized":"2023-10-30T12:45:19.125429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:17:21.758467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070370+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:19.070370+0000","last_peered":"2023-10-30T12:45:19.070370+0000","last_clean":"2023-10-30T12:45:19.070370+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:19.070370+0000","last_undegraded":"2023-10-30T12:45:19.070370+0000","last_fullsized":"2023-10-30T12:45:19.070370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125233+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:19.125233+0000","last_peered":"2023-10-30T12:45:19.125233+0000","last_clean":"2023-10-30T12:45:19.125233+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:19.125233+0000","last_undegraded":"2023-10-30T12:45:19.125233+0000","last_fullsized":"2023-10-30T12:45:19.125233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:10:00.599467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074783+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:19.074783+0000","last_peered":"2023-10-30T12:45:19.074783+0000","last_clean":"2023-10-30T12:45:19.074783+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:19.074783+0000","last_undegraded":"2023-10-30T12:45:19.074783+0000","last_fullsized":"2023-10-30T12:45:19.074783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125703+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:19.125703+0000","last_peered":"2023-10-30T12:45:19.125703+0000","last_clean":"2023-10-30T12:45:19.125703+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:19.125703+0000","last_undegraded":"2023-10-30T12:45:19.125703+0000","last_fullsized":"2023-10-30T12:45:19.125703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:08.273130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067586+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:19.067586+0000","last_peered":"2023-10-30T12:45:19.067586+0000","last_clean":"2023-10-30T12:45:19.067586+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:19.067586+0000","last_undegraded":"2023-10-30T12:45:19.067586+0000","last_fullsized":"2023-10-30T12:45:19.067586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.134283+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:19.134283+0000","last_peered":"2023-10-30T12:45:19.134283+0000","last_clean":"2023-10-30T12:45:19.134283+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:19.134283+0000","last_undegraded":"2023-10-30T12:45:19.134283+0000","last_fullsized":"2023-10-30T12:45:19.134283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:59:21.306649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.075237+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:19.075237+0000","last_peered":"2023-10-30T12:45:19.075237+0000","last_clean":"2023-10-30T12:45:19.075237+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:19.075237+0000","last_undegraded":"2023-10-30T12:45:19.075237+0000","last_fullsized":"2023-10-30T12:45:19.075237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131379+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:19.131379+0000","last_peered":"2023-10-30T12:45:19.131379+0000","last_clean":"2023-10-30T12:45:19.131379+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:19.131379+0000","last_undegraded":"2023-10-30T12:45:19.131379+0000","last_fullsized":"2023-10-30T12:45:19.131379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:19.213457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-30T12:45:18.103394+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:18.103394+0000","last_peered":"2023-10-30T12:45:18.103394+0000","last_clean":"2023-10-30T12:45:18.103394+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:18.103394+0000","last_undegraded":"2023-10-30T12:45:18.103394+0000","last_fullsized":"2023-10-30T12:45:18.103394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.087815+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:19.087815+0000","last_peered":"2023-10-30T12:45:19.087815+0000","last_clean":"2023-10-30T12:45:19.087815+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:19.087815+0000","last_undegraded":"2023-10-30T12:45:19.087815+0000","last_fullsized":"2023-10-30T12:45:19.087815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.125763+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:19.125763+0000","last_peered":"2023-10-30T12:45:19.125763+0000","last_clean":"2023-10-30T12:45:19.125763+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:19.125763+0000","last_undegraded":"2023-10-30T12:45:19.125763+0000","last_fullsized":"2023-10-30T12:45:19.125763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:12.458726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070897+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:19.070897+0000","last_peered":"2023-10-30T12:45:19.070897+0000","last_clean":"2023-10-30T12:45:19.070897+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:19.070897+0000","last_undegraded":"2023-10-30T12:45:19.070897+0000","last_fullsized":"2023-10-30T12:45:19.070897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.123233+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:19.123233+0000","last_peered":"2023-10-30T12:45:19.123233+0000","last_clean":"2023-10-30T12:45:19.123233+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:19.123233+0000","last_undegraded":"2023-10-30T12:45:19.123233+0000","last_fullsized":"2023-10-30T12:45:19.123233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:45:47.965390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074588+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:19.074588+0000","last_peered":"2023-10-30T12:45:19.074588+0000","last_clean":"2023-10-30T12:45:19.074588+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:19.074588+0000","last_undegraded":"2023-10-30T12:45:19.074588+0000","last_fullsized":"2023-10-30T12:45:19.074588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.076603+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:19.076603+0000","last_peered":"2023-10-30T12:45:19.076603+0000","last_clean":"2023-10-30T12:45:19.076603+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:19.076603+0000","last_undegraded":"2023-10-30T12:45:19.076603+0000","last_fullsized":"2023-10-30T12:45:19.076603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:26:18.509502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.104036+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:19.104036+0000","last_peered":"2023-10-30T12:45:19.104036+0000","last_clean":"2023-10-30T12:45:19.104036+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:19.104036+0000","last_undegraded":"2023-10-30T12:45:19.104036+0000","last_fullsized":"2023-10-30T12:45:19.104036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129905+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:19.129905+0000","last_peered":"2023-10-30T12:45:19.129905+0000","last_clean":"2023-10-30T12:45:19.129905+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:19.129905+0000","last_undegraded":"2023-10-30T12:45:19.129905+0000","last_fullsized":"2023-10-30T12:45:19.129905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:27.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066874+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.066874+0000","last_peered":"2023-10-30T12:45:19.066874+0000","last_clean":"2023-10-30T12:45:19.066874+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.066874+0000","last_undegraded":"2023-10-30T12:45:19.066874+0000","last_fullsized":"2023-10-30T12:45:19.066874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.131160+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:19.131160+0000","last_peered":"2023-10-30T12:45:19.131160+0000","last_clean":"2023-10-30T12:45:19.131160+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:19.131160+0000","last_undegraded":"2023-10-30T12:45:19.131160+0000","last_fullsized":"2023-10-30T12:45:19.131160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:29:23.885208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.079461+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:19.079461+0000","last_peered":"2023-10-30T12:45:19.079461+0000","last_clean":"2023-10-30T12:45:19.079461+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:19.079461+0000","last_undegraded":"2023-10-30T12:45:19.079461+0000","last_fullsized":"2023-10-30T12:45:19.079461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.081681+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:19.081681+0000","last_peered":"2023-10-30T12:45:19.081681+0000","last_clean":"2023-10-30T12:45:19.081681+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:19.081681+0000","last_undegraded":"2023-10-30T12:45:19.081681+0000","last_fullsized":"2023-10-30T12:45:19.081681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:43:34.496700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066804+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:19.066804+0000","last_peered":"2023-10-30T12:45:19.066804+0000","last_clean":"2023-10-30T12:45:19.066804+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:19.066804+0000","last_undegraded":"2023-10-30T12:45:19.066804+0000","last_fullsized":"2023-10-30T12:45:19.066804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.113494+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:19.113494+0000","last_peered":"2023-10-30T12:45:19.113494+0000","last_clean":"2023-10-30T12:45:19.113494+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:19.113494+0000","last_undegraded":"2023-10-30T12:45:19.113494+0000","last_fullsized":"2023-10-30T12:45:19.113494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:55.006512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.074930+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:19.074930+0000","last_peered":"2023-10-30T12:45:19.074930+0000","last_clean":"2023-10-30T12:45:19.074930+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:19.074930+0000","last_undegraded":"2023-10-30T12:45:19.074930+0000","last_fullsized":"2023-10-30T12:45:19.074930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.097149+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:19.097149+0000","last_peered":"2023-10-30T12:45:19.097149+0000","last_clean":"2023-10-30T12:45:19.097149+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:19.097149+0000","last_undegraded":"2023-10-30T12:45:19.097149+0000","last_fullsized":"2023-10-30T12:45:19.097149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:50:30.460567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.070532+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:19.070532+0000","last_peered":"2023-10-30T12:45:19.070532+0000","last_clean":"2023-10-30T12:45:19.070532+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:19.070532+0000","last_undegraded":"2023-10-30T12:45:19.070532+0000","last_fullsized":"2023-10-30T12:45:19.070532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.117315+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:19.117315+0000","last_peered":"2023-10-30T12:45:19.117315+0000","last_clean":"2023-10-30T12:45:19.117315+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:19.117315+0000","last_undegraded":"2023-10-30T12:45:19.117315+0000","last_fullsized":"2023-10-30T12:45:19.117315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:22:50.870760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.067994+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:19.067994+0000","last_peered":"2023-10-30T12:45:19.067994+0000","last_clean":"2023-10-30T12:45:19.067994+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:19.067994+0000","last_undegraded":"2023-10-30T12:45:19.067994+0000","last_fullsized":"2023-10-30T12:45:19.067994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.078936+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:19.078936+0000","last_peered":"2023-10-30T12:45:19.078936+0000","last_clean":"2023-10-30T12:45:19.078936+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:19.078936+0000","last_undegraded":"2023-10-30T12:45:19.078936+0000","last_fullsized":"2023-10-30T12:45:19.078936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:27.368058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.066164+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:19.066164+0000","last_peered":"2023-10-30T12:45:19.066164+0000","last_clean":"2023-10-30T12:45:19.066164+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:19.066164+0000","last_undegraded":"2023-10-30T12:45:19.066164+0000","last_fullsized":"2023-10-30T12:45:19.066164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.099909+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:19.099909+0000","last_peered":"2023-10-30T12:45:19.099909+0000","last_clean":"2023-10-30T12:45:19.099909+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:19.099909+0000","last_undegraded":"2023-10-30T12:45:19.099909+0000","last_fullsized":"2023-10-30T12:45:19.099909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:38:28.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.081123+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:19.081123+0000","last_peered":"2023-10-30T12:45:19.081123+0000","last_clean":"2023-10-30T12:45:19.081123+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:19.081123+0000","last_undegraded":"2023-10-30T12:45:19.081123+0000","last_fullsized":"2023-10-30T12:45:19.081123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.102712+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:19.102712+0000","last_peered":"2023-10-30T12:45:19.102712+0000","last_clean":"2023-10-30T12:45:19.102712+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:19.102712+0000","last_undegraded":"2023-10-30T12:45:19.102712+0000","last_fullsized":"2023-10-30T12:45:19.102712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:53:50.701760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.135999+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:19.135999+0000","last_peered":"2023-10-30T12:45:19.135999+0000","last_clean":"2023-10-30T12:45:19.135999+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:19.135999+0000","last_undegraded":"2023-10-30T12:45:19.135999+0000","last_fullsized":"2023-10-30T12:45:19.135999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:51:55.133787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071737+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:19.071737+0000","last_peered":"2023-10-30T12:45:19.071737+0000","last_clean":"2023-10-30T12:45:19.071737+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:19.071737+0000","last_undegraded":"2023-10-30T12:45:19.071737+0000","last_fullsized":"2023-10-30T12:45:19.071737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.135971+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:19.135971+0000","last_peered":"2023-10-30T12:45:19.135971+0000","last_clean":"2023-10-30T12:45:19.135971+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:19.135971+0000","last_undegraded":"2023-10-30T12:45:19.135971+0000","last_fullsized":"2023-10-30T12:45:19.135971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:00:27.039964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.079089+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:19.079089+0000","last_peered":"2023-10-30T12:45:19.079089+0000","last_clean":"2023-10-30T12:45:19.079089+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:19.079089+0000","last_undegraded":"2023-10-30T12:45:19.079089+0000","last_fullsized":"2023-10-30T12:45:19.079089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.132171+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:19.132171+0000","last_peered":"2023-10-30T12:45:19.132171+0000","last_clean":"2023-10-30T12:45:19.132171+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:19.132171+0000","last_undegraded":"2023-10-30T12:45:19.132171+0000","last_fullsized":"2023-10-30T12:45:19.132171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:16.015616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.072170+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:19.072170+0000","last_peered":"2023-10-30T12:45:19.072170+0000","last_clean":"2023-10-30T12:45:19.072170+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:19.072170+0000","last_undegraded":"2023-10-30T12:45:19.072170+0000","last_fullsized":"2023-10-30T12:45:19.072170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.129361+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:19.129361+0000","last_peered":"2023-10-30T12:45:19.129361+0000","last_clean":"2023-10-30T12:45:19.129361+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:19.129361+0000","last_undegraded":"2023-10-30T12:45:19.129361+0000","last_fullsized":"2023-10-30T12:45:19.129361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:35:54.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.089060+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:19.089060+0000","last_peered":"2023-10-30T12:45:19.089060+0000","last_clean":"2023-10-30T12:45:19.089060+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:19.089060+0000","last_undegraded":"2023-10-30T12:45:19.089060+0000","last_fullsized":"2023-10-30T12:45:19.089060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":648191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":192512,"data_stored":64598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":180224,"data_stored":57823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":653947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:25.612 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-30T12:45:25.613 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-30T12:45:25.613 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-30T12:45:25.613 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-30T12:45:25.614 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-30T12:45:25.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 Processor -- start 2023-10-30T12:45:25.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 -- start start 2023-10-30T12:45:25.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 0x7f8e0015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8e000c42a0 0x7f8e000ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e00159f20 con 0x7f8e00153c80 2023-10-30T12:45:25.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0015dec0 con 0x7f8e000c42a0 2023-10-30T12:45:25.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8e05617640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e001599a0 con 0x7f8e000c4a10 2023-10-30T12:45:25.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dff7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8e000c42a0 0x7f8e000ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dfeffd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8e00153c80 0x7f8e0015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44740/0 (socket says 172.21.15.8:44740) 2023-10-30T12:45:25.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dfeffd640 1 -- 172.21.15.8:0/121052436 learned_addr learned my addr 172.21.15.8:0/121052436 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:25.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8df7fff640 1 --2- 172.21.15.8:0/121052436 >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 0x7f8e0015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dff7fe640 1 -- 172.21.15.8:0/121052436 >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 msgr2=0x7f8e0015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:25.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dff7fe640 1 --2- 172.21.15.8:0/121052436 >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 0x7f8e0015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dff7fe640 1 -- 172.21.15.8:0/121052436 >> v1:172.21.15.8:6789/0 conn(0x7f8e00153c80 legacy=0x7f8e0015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dff7fe640 1 -- 172.21.15.8:0/121052436 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0015d620 con 0x7f8e000c42a0 2023-10-30T12:45:25.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8df7fff640 1 --2- 172.21.15.8:0/121052436 >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 0x7f8e0015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:25.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.793+0000 7f8dfe7fc640 1 -- 172.21.15.8:0/121052436 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736854806 0 0) 0x7f8e00159f20 con 0x7f8e00153c80 2023-10-30T12:45:25.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8dff7fe640 1 --2- 172.21.15.8:0/121052436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e000c42a0 0x7f8e000ca1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8e0006be50 tx=0x7f8df801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dabfcaa1636b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8dfe7fc640 1 -- 172.21.15.8:0/121052436 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80b0440 con 0x7f8e000c42a0 2023-10-30T12:45:25.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8dfe7fc640 1 -- 172.21.15.8:0/121052436 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df80b05a0 con 0x7f8e000c42a0 2023-10-30T12:45:25.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8dfe7fc640 1 -- 172.21.15.8:0/121052436 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80b0930 con 0x7f8e000c42a0 2023-10-30T12:45:25.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 -- 172.21.15.8:0/121052436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e000c42a0 msgr2=0x7f8e000ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 --2- 172.21.15.8:0/121052436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e000c42a0 0x7f8e000ca1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8e0006be50 tx=0x7f8df801f0f0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 -- 172.21.15.8:0/121052436 shutdown_connections 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 --2- 172.21.15.8:0/121052436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e000c42a0 0x7f8e000ca1e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 --2- 172.21.15.8:0/121052436 >> v2:172.21.15.46:3300/0 conn(0x7f8e000c4a10 0x7f8e0015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 -- 172.21.15.8:0/121052436 >> 172.21.15.8:0/121052436 conn(0x7f8e000715c0 msgr2=0x7f8e000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:25.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 -- 172.21.15.8:0/121052436 shutdown_connections 2023-10-30T12:45:25.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.797+0000 7f8e05617640 1 -- 172.21.15.8:0/121052436 wait complete. 2023-10-30T12:45:25.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 Processor -- start 2023-10-30T12:45:25.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- start start 2023-10-30T12:45:25.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8e000c42a0 0x7f8e00154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e000c4a10 0x7f8e001506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000ce760 con 0x7f8e00153c80 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00154650 con 0x7f8e000c4a10 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000c9190 con 0x7f8e000c42a0 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36110/0 (socket says 172.21.15.8:36110) 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 -- 172.21.15.8:0/703943274 learned_addr learned my addr 172.21.15.8:0/703943274 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:25.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8df7fff640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e000c4a10 0x7f8e001506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dfeffd640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8e000c42a0 0x7f8e00154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 -- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8e000c42a0 msgr2=0x7f8e00154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8e000c42a0 0x7f8e00154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 -- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e000c4a10 msgr2=0x7f8e001506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e000c4a10 0x7f8e001506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:25.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 -- 172.21.15.8:0/703943274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8009a70 con 0x7f8e00153c80 2023-10-30T12:45:25.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8dff7fe640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8df801cd20 tx=0x7f8df801be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80b0440 con 0x7f8e00153c80 2023-10-30T12:45:25.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df80ba820 con 0x7f8e00153c80 2023-10-30T12:45:25.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80babd0 con 0x7f8e00153c80 2023-10-30T12:45:25.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000c9470 con 0x7f8e00153c80 2023-10-30T12:45:25.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.801+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e000cec90 con 0x7f8e00153c80 2023-10-30T12:45:25.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00154ac0 con 0x7f8e00153c80 2023-10-30T12:45:25.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8df80b7050 con 0x7f8e00153c80 2023-10-30T12:45:25.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8df77fe640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:25.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df8122600 con 0x7f8e00153c80 2023-10-30T12:45:25.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8dfeffd640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:25.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.805+0000 7f8dfeffd640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 0x7f8de0068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f70 tx=0x7f8df0007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:25.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:25.809+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8e00154ac0 con 0x7f8e00153c80 2023-10-30T12:45:26.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.001+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8e00154ac0 con 0x7f8e00153c80 2023-10-30T12:45:26.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.005+0000 7f8df77fe640 1 -- 172.21.15.8:0/703943274 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164422 (secure 0 0 0) 0x7f8e00154ac0 con 0x7f8e00153c80 2023-10-30T12:45:26.009 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:26.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 msgr2=0x7f8de0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 0x7f8de0068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f70 tx=0x7f8df0007c10 comp rx=0 tx=0).stop 2023-10-30T12:45:26.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 msgr2=0x7f8e00150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8df801cd20 tx=0x7f8df801be90 comp rx=0 tx=0).stop 2023-10-30T12:45:26.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 shutdown_connections 2023-10-30T12:45:26.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8de00664a0 0x7f8de0068960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e000c4a10 0x7f8e001506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e00153c80 0x7f8e00150c20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 --2- 172.21.15.8:0/703943274 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8e000c42a0 0x7f8e00154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 >> 172.21.15.8:0/703943274 conn(0x7f8e000715c0 msgr2=0x7f8e000719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 shutdown_connections 2023-10-30T12:45:26.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.009+0000 7f8e05617640 1 -- 172.21.15.8:0/703943274 wait complete. 2023-10-30T12:45:26.029 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":5,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2382673595},{"type":"v1","addr":"172.21.15.8:6833","nonce":2382673595}]},"active_addr":"172.21.15.8:6833/2382673595","active_change":"2023-10-30T12:44:57.594572+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":608960973}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":2862318168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3050527705}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":2979091858}]}]} 2023-10-30T12:45:26.033 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-30T12:45:26.033 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-30T12:45:26.033 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:26.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.173+0000 7f260f49c640 1 Processor -- start 2023-10-30T12:45:26.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.173+0000 7f260f49c640 1 -- start start 2023-10-30T12:45:26.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260f49c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260f49c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26080cb8b0 0x7f260806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260f49c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080c5030 con 0x7f2608151c20 2023-10-30T12:45:26.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260f49c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2608157830 con 0x7f26080cb8b0 2023-10-30T12:45:26.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260f49c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608157b10 con 0x7f2608151840 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260d238640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26080cb8b0 0x7f260806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260d238640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33250/0 (socket says 172.21.15.8:33250) 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260d238640 1 -- 172.21.15.8:0/2187998055 learned_addr learned my addr 172.21.15.8:0/2187998055 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26080cb8b0 0x7f260806b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49888/0 (socket says 172.21.15.8:49888) 2023-10-30T12:45:26.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260ca37640 1 --1- 172.21.15.8:0/2187998055 >> v1:172.21.15.46:6789/0 conn(0x7f2608151c20 0x7f2608072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:32970/0 (socket says 172.21.15.8:32970) 2023-10-30T12:45:26.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 -- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 msgr2=0x7f26080c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 -- 172.21.15.8:0/2187998055 >> v1:172.21.15.46:6789/0 conn(0x7f2608151c20 legacy=0x7f2608072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260da39640 1 -- 172.21.15.8:0/2187998055 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608157df0 con 0x7f26080cb8b0 2023-10-30T12:45:26.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.177+0000 7f260d238640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:26.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260da39640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080cb8b0 0x7f260806b600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260001f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d728a269fd05152 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f25fffff640 1 -- 172.21.15.8:0/2187998055 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000af600 con 0x7f26080cb8b0 2023-10-30T12:45:26.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f25fffff640 1 -- 172.21.15.8:0/2187998055 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26000ab050 con 0x7f26080cb8b0 2023-10-30T12:45:26.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f25fffff640 1 -- 172.21.15.8:0/2187998055 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000af980 con 0x7f26080cb8b0 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 -- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080cb8b0 msgr2=0x7f260806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080cb8b0 0x7f260806b600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260001f130 comp rx=0 tx=0).stop 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 -- 172.21.15.8:0/2187998055 shutdown_connections 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080cb8b0 0x7f260806b600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 --2- 172.21.15.8:0/2187998055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151840 0x7f26080c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 -- 172.21.15.8:0/2187998055 >> 172.21.15.8:0/2187998055 conn(0x7f26080cac70 msgr2=0x7f26080cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 -- 172.21.15.8:0/2187998055 shutdown_connections 2023-10-30T12:45:26.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 -- 172.21.15.8:0/2187998055 wait complete. 2023-10-30T12:45:26.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.181+0000 7f260f49c640 1 Processor -- start 2023-10-30T12:45:26.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- start start 2023-10-30T12:45:26.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2608151840 0x7f2608070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151c20 0x7f2608070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608153d80 con 0x7f26080cb8b0 2023-10-30T12:45:26.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081eeec0 con 0x7f2608151840 2023-10-30T12:45:26.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26081ef890 con 0x7f2608151c20 2023-10-30T12:45:26.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36118/0 (socket says 172.21.15.8:36118) 2023-10-30T12:45:26.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 -- 172.21.15.8:0/3794191648 learned_addr learned my addr 172.21.15.8:0/3794191648 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:26.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260ca37640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2608151840 0x7f2608070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260da39640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151c20 0x7f2608070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 -- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151c20 msgr2=0x7f2608070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151c20 0x7f2608070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 -- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2608151840 msgr2=0x7f2608070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2608151840 0x7f2608070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 -- 172.21.15.8:0/3794191648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040094d0 con 0x7f26080cb8b0 2023-10-30T12:45:26.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260d238640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f260400ea00 tx=0x7f260400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400ca60 con 0x7f26080cb8b0 2023-10-30T12:45:26.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400cbc0 con 0x7f26080cb8b0 2023-10-30T12:45:26.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604013470 con 0x7f26080cb8b0 2023-10-30T12:45:26.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2600009a70 con 0x7f26080cb8b0 2023-10-30T12:45:26.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081ee890 con 0x7f26080cb8b0 2023-10-30T12:45:26.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.185+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260804e170 con 0x7f26080cb8b0 2023-10-30T12:45:26.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.189+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f26040135d0 con 0x7f26080cb8b0 2023-10-30T12:45:26.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.189+0000 7f25fdffb640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 0x7f25ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.189+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2604084270 con 0x7f26080cb8b0 2023-10-30T12:45:26.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.189+0000 7f260ca37640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 0x7f25ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.189+0000 7f260ca37640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 0x7f25ec068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26081e91b0 tx=0x7f25f8007cf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.193+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2604051650 con 0x7f26080cb8b0 2023-10-30T12:45:26.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.349+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26080c48c0 con 0x7f26080cb8b0 2023-10-30T12:45:26.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.349+0000 7f25fdffb640 1 -- 172.21.15.8:0/3794191648 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f2604054c70 con 0x7f26080cb8b0 2023-10-30T12:45:26.354 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:26.354 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:26.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 msgr2=0x7f25ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 0x7f25ec068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26081e91b0 tx=0x7f25f8007cf0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 msgr2=0x7f26081ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f260400ea00 tx=0x7f260400eec0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 shutdown_connections 2023-10-30T12:45:26.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25ec0664a0 0x7f25ec068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2608151840 0x7f2608070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2608151c20 0x7f2608070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 --2- 172.21.15.8:0/3794191648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26080cb8b0 0x7f26081ef350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 >> 172.21.15.8:0/3794191648 conn(0x7f26080cac70 msgr2=0x7f26081503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 shutdown_connections 2023-10-30T12:45:26.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.353+0000 7f260f49c640 1 -- 172.21.15.8:0/3794191648 wait complete. 2023-10-30T12:45:26.374 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-30T12:45:26.375 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:26.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.513+0000 7efcc193e640 1 Processor -- start 2023-10-30T12:45:26.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 -- start start 2023-10-30T12:45:26.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc153c80 0x7efcbc15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efcbc0c4a10 0x7efcbc0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc159f20 con 0x7efcbc15a3b0 2023-10-30T12:45:26.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc15dec0 con 0x7efcbc0c4a10 2023-10-30T12:45:26.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcc193e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1599a0 con 0x7efcbc153c80 2023-10-30T12:45:26.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efcbc0c4a10 0x7efcbc0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcbb7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7efcbc15a3b0 0x7efcbc0ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44750/0 (socket says 172.21.15.8:44750) 2023-10-30T12:45:26.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efcbc0c4a10 0x7efcbc0c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36132/0 (socket says 172.21.15.8:36132) 2023-10-30T12:45:26.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcbb7fe640 1 -- 172.21.15.8:0/434498140 learned_addr learned my addr 172.21.15.8:0/434498140 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:26.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcbaffd640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc153c80 0x7efcbc15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 -- 172.21.15.8:0/434498140 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc153c80 msgr2=0x7efcbc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc153c80 0x7efcbc15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 -- 172.21.15.8:0/434498140 >> v1:172.21.15.8:6789/0 conn(0x7efcbc15a3b0 legacy=0x7efcbc0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcb3fff640 1 -- 172.21.15.8:0/434498140 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0c47e0 con 0x7efcbc0c4a10 2023-10-30T12:45:26.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.517+0000 7efcba7fc640 1 -- 172.21.15.8:0/434498140 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407404398 0 0) 0x7efcbc159f20 con 0x7efcbc15a3b0 2023-10-30T12:45:26.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcb3fff640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0c42a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efca40097c0 tx=0x7efca401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f695ea5b7e466005 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcba7fc640 1 -- 172.21.15.8:0/434498140 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40a8970 con 0x7efcbc0c4a10 2023-10-30T12:45:26.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcba7fc640 1 -- 172.21.15.8:0/434498140 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40a8ad0 con 0x7efcbc0c4a10 2023-10-30T12:45:26.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcba7fc640 1 -- 172.21.15.8:0/434498140 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40a8e60 con 0x7efcbc0c4a10 2023-10-30T12:45:26.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- 172.21.15.8:0/434498140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc0c4a10 msgr2=0x7efcbc0c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0c42a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efca40097c0 tx=0x7efca401ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:26.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- 172.21.15.8:0/434498140 shutdown_connections 2023-10-30T12:45:26.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0c42a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 --2- 172.21.15.8:0/434498140 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc153c80 0x7efcbc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- 172.21.15.8:0/434498140 >> 172.21.15.8:0/434498140 conn(0x7efcbc0715c0 msgr2=0x7efcbc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- 172.21.15.8:0/434498140 shutdown_connections 2023-10-30T12:45:26.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- 172.21.15.8:0/434498140 wait complete. 2023-10-30T12:45:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 Processor -- start 2023-10-30T12:45:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.521+0000 7efcc193e640 1 -- start start 2023-10-30T12:45:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc15a3b0 0x7efcbc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b4070 con 0x7efcbc153c80 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b3180 con 0x7efcbc0c4a10 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b3460 con 0x7efcbc15a3b0 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36134/0 (socket says 172.21.15.8:36134) 2023-10-30T12:45:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 -- 172.21.15.8:0/622375216 learned_addr learned my addr 172.21.15.8:0/622375216 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcbb7fe640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc15a3b0 0x7efcbc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcbaffd640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 -- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc15a3b0 msgr2=0x7efcbc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc15a3b0 0x7efcbc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 -- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc0c4a10 msgr2=0x7efcbc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 -- 172.21.15.8:0/622375216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4009420 con 0x7efcbc153c80 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb3fff640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efca40b3040 tx=0x7efca401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b9630 con 0x7efcbc153c80 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40b97b0 con 0x7efcbc153c80 2023-10-30T12:45:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b9b60 con 0x7efcbc153c80 2023-10-30T12:45:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0b3740 con 0x7efcbc153c80 2023-10-30T12:45:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc0b3d00 con 0x7efcbc153c80 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.525+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc04e170 con 0x7efcbc153c80 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.529+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efca40b7040 con 0x7efcbc153c80 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.529+0000 7efcb8ff9640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 0x7efc9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.529+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efca4121f10 con 0x7efcbc153c80 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.529+0000 7efcbaffd640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 0x7efc9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.529+0000 7efcbaffd640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 0x7efc9c068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcac009af0 tx=0x7efcac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.533+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efca40ef380 con 0x7efcbc153c80 2023-10-30T12:45:26.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.689+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcbc0b1b20 con 0x7efcbc153c80 2023-10-30T12:45:26.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcb8ff9640 1 -- 172.21.15.8:0/622375216 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7efca40b5070 con 0x7efcbc153c80 2023-10-30T12:45:26.694 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:26.695 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:26.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 msgr2=0x7efc9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 0x7efc9c068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcac009af0 tx=0x7efcac008040 comp rx=0 tx=0).stop 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 msgr2=0x7efcbc0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efca40b3040 tx=0x7efca401ba10 comp rx=0 tx=0).stop 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 shutdown_connections 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc9c0664a0 0x7efc9c068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efcbc15a3b0 0x7efcbc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcbc153c80 0x7efcbc0b4980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 --2- 172.21.15.8:0/622375216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcbc0c4a10 0x7efcbc0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 >> 172.21.15.8:0/622375216 conn(0x7efcbc0715c0 msgr2=0x7efcbc157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 shutdown_connections 2023-10-30T12:45:26.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.693+0000 7efcc193e640 1 -- 172.21.15.8:0/622375216 wait complete. 2023-10-30T12:45:26.716 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-30T12:45:26.716 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-30T12:45:26.717 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-30T12:45:26.717 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-30T12:45:26.718 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-30T12:45:26.718 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-30T12:45:26.719 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-30T12:45:26.719 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-30T12:45:26.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.957+0000 7f777ae46640 1 Processor -- start 2023-10-30T12:45:26.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 -- start start 2023-10-30T12:45:26.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb586d65640 1 Processor -- start 2023-10-30T12:45:26.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb586d65640 1 -- start start 2023-10-30T12:45:26.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb586d65640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5780452c0 0x7fb5780456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb586d65640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb5780e6760 0x7fb5780e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0dd430 0x7f776c04b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f776c0452c0 0x7f776c0456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f776c0e6760 0x7f776c0e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c04bac0 con 0x7f776c0452c0 2023-10-30T12:45:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c007580 con 0x7f776c0e6760 2023-10-30T12:45:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f777ae46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c04b830 con 0x7f776c0dd430 2023-10-30T12:45:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb584b01640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fb5780dd430 0x7fb57804b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44764/0 (socket says 172.21.15.8:44764) 2023-10-30T12:45:26.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb57ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5780452c0 0x7fb5780456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7fb57ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5780452c0 0x7fb5780456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49924/0 (socket says 172.21.15.8:49924) 2023-10-30T12:45:26.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f77793e3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f776c0e6760 0x7f776c0e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f77793e3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f776c0e6760 0x7f776c0e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49918/0 (socket says 172.21.15.8:49918) 2023-10-30T12:45:26.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.961+0000 7f77793e3640 1 -- 172.21.15.8:0/43181902 learned_addr learned my addr 172.21.15.8:0/43181902 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:26.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb586d65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb578007580 con 0x7fb5780dd430 2023-10-30T12:45:26.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb586d65640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57804bac0 con 0x7fb5780e6760 2023-10-30T12:45:26.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb586d65640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb578045c30 con 0x7fb5780452c0 2023-10-30T12:45:26.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb584b01640 1 -- 172.21.15.8:0/3933365218 learned_addr learned my addr 172.21.15.8:0/3933365218 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:26.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7f77793e3640 1 -- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0dd430 msgr2=0x7f776c04b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb57ffff640 1 -- 172.21.15.8:0/3933365218 >> v2:172.21.15.8:3300/0 conn(0x7fb5780e6760 msgr2=0x7fb5780e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb57ffff640 1 --2- 172.21.15.8:0/3933365218 >> v2:172.21.15.8:3300/0 conn(0x7fb5780e6760 0x7fb5780e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb57ffff640 1 -- 172.21.15.8:0/3933365218 >> v1:172.21.15.8:6789/0 conn(0x7fb5780dd430 legacy=0x7fb57804b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7fb57ffff640 1 -- 172.21.15.8:0/3933365218 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578006970 con 0x7fb5780452c0 2023-10-30T12:45:26.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7f77793e3640 1 --2- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0dd430 0x7f776c04b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7f77793e3640 1 -- 172.21.15.8:0/43181902 >> v2:172.21.15.8:3300/0 conn(0x7f776c0452c0 msgr2=0x7f776c0456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7f77793e3640 1 --2- 172.21.15.8:0/43181902 >> v2:172.21.15.8:3300/0 conn(0x7f776c0452c0 0x7f776c0456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.965+0000 7f77793e3640 1 -- 172.21.15.8:0/43181902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c006970 con 0x7f776c0e6760 2023-10-30T12:45:26.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f77793e3640 1 --2- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0e6760 0x7f776c0e6bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f776c009500 tx=0x7f776001ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7dde93f94be9f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7fb57ffff640 1 --2- 172.21.15.8:0/3933365218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780452c0 0x7fb5780456f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb578050fe0 tx=0x7fb56c01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb86b0e1017607c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f77737fe640 1 -- 172.21.15.8:0/43181902 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600a89d0 con 0x7f776c0e6760 2023-10-30T12:45:26.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f77737fe640 1 -- 172.21.15.8:0/43181902 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77600a8b30 con 0x7f776c0e6760 2023-10-30T12:45:26.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f77737fe640 1 -- 172.21.15.8:0/43181902 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600b0440 con 0x7f776c0e6760 2023-10-30T12:45:26.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7fb57f7fe640 1 -- 172.21.15.8:0/3933365218 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0a8930 con 0x7fb5780452c0 2023-10-30T12:45:26.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7fb57f7fe640 1 -- 172.21.15.8:0/3933365218 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0a8a90 con 0x7fb5780452c0 2023-10-30T12:45:26.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7fb57f7fe640 1 -- 172.21.15.8:0/3933365218 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0a8e20 con 0x7fb5780452c0 2023-10-30T12:45:26.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f777ae46640 1 -- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0e6760 msgr2=0x7f776c0e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f777ae46640 1 --2- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0e6760 0x7f776c0e6bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f776c009500 tx=0x7f776001ed10 comp rx=0 tx=0).stop 2023-10-30T12:45:26.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 -- 172.21.15.8:0/43181902 shutdown_connections 2023-10-30T12:45:26.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 --2- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0e6760 0x7f776c0e6bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 --2- 172.21.15.8:0/43181902 >> v2:172.21.15.8:3300/0 conn(0x7f776c0452c0 0x7f776c0456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f67b7bd3640 1 Processor -- start 2023-10-30T12:45:26.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.969+0000 7f67b7bd3640 1 -- start start 2023-10-30T12:45:26.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b7bd3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f67b006b760 0x7f67b015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7fb586d65640 1 -- 172.21.15.8:0/3933365218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780452c0 msgr2=0x7fb5780456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7fb586d65640 1 --2- 172.21.15.8:0/3933365218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780452c0 0x7fb5780456f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb578050fe0 tx=0x7fb56c01ec30 comp rx=0 tx=0).stop 2023-10-30T12:45:26.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 --2- 172.21.15.8:0/43181902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0dd430 0x7f776c04b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 -- 172.21.15.8:0/43181902 >> 172.21.15.8:0/43181902 conn(0x7f776c050170 msgr2=0x7f776c050580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b7bd3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b006c420 0x7f67b00c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 -- 172.21.15.8:0/43181902 shutdown_connections 2023-10-30T12:45:26.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f777ae46640 1 -- 172.21.15.8:0/43181902 wait complete. 2023-10-30T12:45:26.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b7bd3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b0071e00 con 0x7f67b006bb40 2023-10-30T12:45:26.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b7bd3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b0070020 con 0x7f67b006b760 2023-10-30T12:45:26.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b7bd3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b015eb90 con 0x7f67b006c420 2023-10-30T12:45:26.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b516e640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f67b006bb40 0x7f67b0157a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44772/0 (socket says 172.21.15.8:44772) 2023-10-30T12:45:26.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b516e640 1 -- 172.21.15.8:0/1100198357 learned_addr learned my addr 172.21.15.8:0/1100198357 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:26.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b6170640 1 --2- 172.21.15.8:0/1100198357 >> v2:172.21.15.46:3300/0 conn(0x7f67b006c420 0x7f67b00c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 -- 172.21.15.8:0/3933365218 shutdown_connections 2023-10-30T12:45:26.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 --2- 172.21.15.8:0/3933365218 >> v2:172.21.15.8:3300/0 conn(0x7fb5780e6760 0x7fb5780e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 --2- 172.21.15.8:0/3933365218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780452c0 0x7fb5780456f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 -- 172.21.15.8:0/3933365218 >> 172.21.15.8:0/3933365218 conn(0x7fb578050170 msgr2=0x7fb578050580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 -- 172.21.15.8:0/3933365218 shutdown_connections 2023-10-30T12:45:26.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 Processor -- start 2023-10-30T12:45:26.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 -- 172.21.15.8:0/3933365218 wait complete. 2023-10-30T12:45:26.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 -- start start 2023-10-30T12:45:26.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b6170640 1 -- 172.21.15.8:0/1100198357 >> v2:172.21.15.8:3300/0 conn(0x7f67b006b760 msgr2=0x7f67b015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b6170640 1 --2- 172.21.15.8:0/1100198357 >> v2:172.21.15.8:3300/0 conn(0x7f67b006b760 0x7f67b015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b6170640 1 -- 172.21.15.8:0/1100198357 >> v1:172.21.15.8:6789/0 conn(0x7f67b006bb40 legacy=0x7f67b0157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:26.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.973+0000 7f67b6170640 1 -- 172.21.15.8:0/1100198357 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0070f10 con 0x7f67b006c420 2023-10-30T12:45:26.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 0x7f776c035500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0dd430 0x7f776c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f67b6170640 1 --2- 172.21.15.8:0/1100198357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006c420 0x7f67b00c57a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67b015b7a0 tx=0x7f67a801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cba4f7ce489b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0e6760 0x7f776c0e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7778be2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 0x7f776c035500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c04fb00 con 0x7f776c0452c0 2023-10-30T12:45:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0daf30 con 0x7f776c0dd430 2023-10-30T12:45:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f777ae46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0dc570 con 0x7f776c0e6760 2023-10-30T12:45:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7778be2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 0x7f776c035500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36172/0 (socket says 172.21.15.8:36172) 2023-10-30T12:45:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7778be2640 1 -- 172.21.15.8:0/848749622 learned_addr learned my addr 172.21.15.8:0/848749622 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:26.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7773fff640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0dd430 0x7f776c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0e6760 0x7f776c0e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:26.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 Processor -- start 2023-10-30T12:45:26.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7fb586d65640 1 -- start start 2023-10-30T12:45:26.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f67b496d640 1 -- 172.21.15.8:0/1100198357 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80a5960 con 0x7f67b006c420 2023-10-30T12:45:26.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f67b496d640 1 -- 172.21.15.8:0/1100198357 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a80a5ac0 con 0x7f67b006c420 2023-10-30T12:45:26.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f67b496d640 1 -- 172.21.15.8:0/1100198357 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80a5960 con 0x7f67b006c420 2023-10-30T12:45:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 -- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0dd430 msgr2=0x7f776c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0dd430 0x7f776c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 -- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 msgr2=0x7f776c035500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 0x7f776c035500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f77793e3640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600094d0 con 0x7f776c0e6760 2023-10-30T12:45:26.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7773fff640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f776c0dd430 0x7f776c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:26.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.977+0000 7f7778be2640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f776c0452c0 0x7f776c035500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 -- 172.21.15.8:0/1100198357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006c420 msgr2=0x7f67b00c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/1100198357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006c420 0x7f67b00c57a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67b015b7a0 tx=0x7f67a801c8c0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 -- 172.21.15.8:0/1100198357 shutdown_connections 2023-10-30T12:45:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/1100198357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006c420 0x7f67b00c57a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/1100198357 >> v2:172.21.15.8:3300/0 conn(0x7f67b006b760 0x7f67b015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 -- 172.21.15.8:0/1100198357 >> 172.21.15.8:0/1100198357 conn(0x7f67b00cac70 msgr2=0x7f67b00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 -- 172.21.15.8:0/1100198357 shutdown_connections 2023-10-30T12:45:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f4a1e09a640 1 Processor -- start 2023-10-30T12:45:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb5780452c0 0x7fb5780e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 0x7fb5780354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f67b7bd3640 1 -- 172.21.15.8:0/1100198357 wait complete. 2023-10-30T12:45:27.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5780e6760 0x7fb578177180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780e0fe0 con 0x7fb5780e6760 2023-10-30T12:45:27.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578176ca0 con 0x7fb5780dd430 2023-10-30T12:45:27.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb586d65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780e6220 con 0x7fb5780452c0 2023-10-30T12:45:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fe97593e640 1 Processor -- start 2023-10-30T12:45:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 0x7fb5780354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 0x7fb5780354e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49938/0 (socket says 172.21.15.8:49938) 2023-10-30T12:45:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 -- 172.21.15.8:0/3892925147 learned_addr learned my addr 172.21.15.8:0/3892925147 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb585302640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5780e6760 0x7fb578177180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f77793e3640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0e6760 0x7f776c0e59f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f77600af000 tx=0x7f776001bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fe97593e640 1 -- start start 2023-10-30T12:45:27.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600b0e00 con 0x7f776c0e6760 2023-10-30T12:45:27.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77600ba440 con 0x7f776c0e6760 2023-10-30T12:45:27.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600ba7f0 con 0x7f776c0e6760 2023-10-30T12:45:27.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 -- 172.21.15.8:0/3892925147 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb5780452c0 msgr2=0x7fb5780e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb5780452c0 0x7fb5780e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 -- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5780e6760 msgr2=0x7fb578177180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5780e6760 0x7fb578177180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7fb57ffff640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009420 con 0x7fb5780dd430 2023-10-30T12:45:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c0dc850 con 0x7f776c0e6760 2023-10-30T12:45:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c034fe0 con 0x7f776c0e6760 2023-10-30T12:45:27.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.981+0000 7f4a1e09a640 1 -- start start 2023-10-30T12:45:27.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a1e09a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4a1806b760 0x7f4a1815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a1e09a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1806c420 0x7f4a180c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a1e09a640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a18071e00 con 0x7f4a1806bb40 2023-10-30T12:45:27.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a1e09a640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a18070020 con 0x7f4a1806b760 2023-10-30T12:45:27.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a1e09a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1815eb90 con 0x7f4a1806c420 2023-10-30T12:45:27.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96effd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9680062b0 0x7fe9680df0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:32980/0 (socket says 172.21.15.8:32980) 2023-10-30T12:45:27.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96e7fc640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe96804c7f0 0x7fe9680df800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44778/0 (socket says 172.21.15.8:44778) 2023-10-30T12:45:27.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1806c420 0x7f4a180c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1806c420 0x7f4a180c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49946/0 (socket says 172.21.15.8:49946) 2023-10-30T12:45:27.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 -- 172.21.15.8:0/1423258953 learned_addr learned my addr 172.21.15.8:0/1423258953 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fb57ffff640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 0x7fb5780354e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb56c01c860 tx=0x7fb56c01b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f776c006f30 con 0x7f776c0e6760 2023-10-30T12:45:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96effd640 1 -- 172.21.15.8:0/1814770627 learned_addr learned my addr 172.21.15.8:0/1814770627 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe97593e640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe968035230 0x7fe9680d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680dff10 con 0x7fe96804c7f0 2023-10-30T12:45:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe968046e00 con 0x7fe9680062b0 2023-10-30T12:45:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe968036400 con 0x7fe968035230 2023-10-30T12:45:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f2e77bc5640 1 Processor -- start 2023-10-30T12:45:27.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0b0e00 con 0x7fb5780dd430 2023-10-30T12:45:27.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0b9400 con 0x7fb5780dd430 2023-10-30T12:45:27.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0b97b0 con 0x7fb5780dd430 2023-10-30T12:45:27.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 -- 172.21.15.8:0/1423258953 >> v2:172.21.15.8:3300/0 conn(0x7f4a1806b760 msgr2=0x7f4a1815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 --2- 172.21.15.8:0/1423258953 >> v2:172.21.15.8:3300/0 conn(0x7f4a1806b760 0x7f4a1815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f2e77bc5640 1 -- start start 2023-10-30T12:45:27.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5780e1230 con 0x7fb5780dd430 2023-10-30T12:45:27.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb578035f70 con 0x7fb5780dd430 2023-10-30T12:45:27.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb578006f30 con 0x7fb5780dd430 2023-10-30T12:45:27.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f77600ba950 con 0x7f776c0e6760 2023-10-30T12:45:27.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7758066570 0x7f7758068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77601226d0 con 0x7f776c0e6760 2023-10-30T12:45:27.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f775806e790 0x7f7758070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7758071260 con 0x7f775806e790 2023-10-30T12:45:27.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7760122a80 con 0x7f776c0e6760 2023-10-30T12:45:27.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f7778be2640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7758066570 0x7f7758068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f7773fff640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f775806e790 0x7f7758070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 Processor -- start 2023-10-30T12:45:27.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 -- start start 2023-10-30T12:45:27.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 0x7f67b015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b006bb40 0x7f67b015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b006c420 0x7f67b00c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00c4820 con 0x7f67b006c420 2023-10-30T12:45:27.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b015ea10 con 0x7f67b006b760 2023-10-30T12:45:27.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b596f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 0x7f67b015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b6170640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b006c420 0x7f67b00c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b596f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 0x7f67b015f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49956/0 (socket says 172.21.15.8:49956) 2023-10-30T12:45:27.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b596f640 1 -- 172.21.15.8:0/3190729618 learned_addr learned my addr 172.21.15.8:0/3190729618 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b015cba0 con 0x7f67b006bb40 2023-10-30T12:45:27.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752541125 0 0) 0x7fe968046e00 con 0x7fe9680062b0 2023-10-30T12:45:27.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe958003530 con 0x7fe9680062b0 2023-10-30T12:45:27.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.1 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947359632 0 0) 0x7fe958003530 con 0x7fe9680062b0 2023-10-30T12:45:27.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe968035230 msgr2=0x7fe9680d5be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:45:27.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe968035230 0x7fe9680d5be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 >> v1:172.21.15.8:6789/0 conn(0x7fe96804c7f0 legacy=0x7fe9680df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680470a0 con 0x7fe9680062b0 2023-10-30T12:45:27.023 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.1 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fe97004e9c0 con 0x7fe9680062b0 2023-10-30T12:45:27.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 >> v1:172.21.15.46:6789/0 conn(0x7fe9680062b0 legacy=0x7fe9680df0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe958003980 0x7fe958003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9580047d0 0x7fe958004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe958009980 0x7fe958005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580147e0 con 0x7fe958003980 2023-10-30T12:45:27.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958009d70 con 0x7fe958009980 2023-10-30T12:45:27.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe958009f30 con 0x7fe9580047d0 2023-10-30T12:45:27.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9580047d0 0x7fe958004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96effd640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe958003980 0x7fe958003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 -- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe958009980 msgr2=0x7fe958005190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:45:27.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe958009980 0x7fe958005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 -- 172.21.15.8:0/1814770627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe958003980 msgr2=0x7fe958003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe958003980 0x7fe958003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96e7fc640 1 -- 172.21.15.8:0/1814770627 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680470a0 con 0x7fe9580047d0 2023-10-30T12:45:27.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7fe96effd640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe958003980 0x7fe958003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:27.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 -- 172.21.15.8:0/1423258953 >> v1:172.21.15.8:6789/0 conn(0x7f4a1806bb40 legacy=0x7f4a18157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.985+0000 7f4a17fff640 1 -- 172.21.15.8:0/1423258953 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18070f10 con 0x7f4a1806c420 2023-10-30T12:45:27.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e77bc5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e700c7110 0x7f2e7006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e77bc5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e7014f910 0x7f2e7006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e77bc5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e701563b0 con 0x7f2e700cb790 2023-10-30T12:45:27.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e77bc5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e700c6030 con 0x7f2e700c7110 2023-10-30T12:45:27.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e77bc5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700c5690 con 0x7f2e7014f910 2023-10-30T12:45:27.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75961640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e700cb790 0x7f2e700c6450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:32986/0 (socket says 172.21.15.8:32986) 2023-10-30T12:45:27.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75961640 1 -- 172.21.15.8:0/1336393415 learned_addr learned my addr 172.21.15.8:0/1336393415 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:27.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75160640 1 --2- 172.21.15.8:0/1336393415 >> v2:172.21.15.46:3300/0 conn(0x7f2e700c7110 0x7f2e7006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e76162640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e7014f910 0x7f2e7006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75160640 1 -- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e7014f910 msgr2=0x7f2e7006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75160640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e7014f910 0x7f2e7006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75160640 1 -- 172.21.15.8:0/1336393415 >> v1:172.21.15.46:6789/0 conn(0x7f2e700cb790 legacy=0x7f2e700c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f2e75160640 1 -- 172.21.15.8:0/1336393415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e700c5970 con 0x7f2e700c7110 2023-10-30T12:45:27.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.989+0000 7f67b516e640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b006bb40 0x7f67b015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 Processor -- start 2023-10-30T12:45:27.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 -- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b006bb40 msgr2=0x7f67b015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 -- start start 2023-10-30T12:45:27.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f7773fff640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f775806e790 0x7f7758070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f7778be2640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7758066570 0x7f7758068a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77740732e0 tx=0x7f7774075690 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b006bb40 0x7f67b015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 -- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b006c420 msgr2=0x7f67b00c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b006c420 0x7f67b00c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009a70 con 0x7f67b006b760 2023-10-30T12:45:27.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b516e640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67b006bb40 0x7f67b015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:27.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b596f640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 0x7f67b015f6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b5b0 tx=0x7f67ac00ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 Processor -- start 2023-10-30T12:45:27.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 -- start start 2023-10-30T12:45:27.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 0x7fb9600ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9c0c1f90 0x7f2a9c0c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0c74c0 con 0x7f2a9c151850 2023-10-30T12:45:27.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0c2900 con 0x7f2a9c06ee90 2023-10-30T12:45:27.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2aa1980640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c06f4d0 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb95f7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 0x7fb9600ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010890 con 0x7f67b006b760 2023-10-30T12:45:27.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac010a10 con 0x7f67b006b760 2023-10-30T12:45:27.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010dc0 con 0x7f67b006b760 2023-10-30T12:45:27.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b0160460 con 0x7f67b006b760 2023-10-30T12:45:27.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00c4440 con 0x7f67b006b760 2023-10-30T12:45:27.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f4a17fff640 1 --2- 172.21.15.8:0/1423258953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c57a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a1815b7a0 tx=0x7f4a0001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e61fae0cbdf90b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 0x7fb9600c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96015e090 con 0x7fb9600c46e0 2023-10-30T12:45:27.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96015de80 con 0x7fb960158000 2023-10-30T12:45:27.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb965ad6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96015ee10 con 0x7fb9601583e0 2023-10-30T12:45:27.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9affd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2a9c151850 0x7f2a9c0c8380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:44790/0 (socket says 172.21.15.8:44790) 2023-10-30T12:45:27.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9affd640 1 -- 172.21.15.8:0/831716468 learned_addr learned my addr 172.21.15.8:0/831716468 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:27.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9b7fe640 1 --2- 172.21.15.8:0/831716468 >> v2:172.21.15.46:3300/0 conn(0x7f2a9c0c1f90 0x7f2a9c0c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f4a167fc640 1 -- 172.21.15.8:0/1423258953 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000aa650 con 0x7f4a1806c420 2023-10-30T12:45:27.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f4a167fc640 1 -- 172.21.15.8:0/1423258953 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000aa7b0 con 0x7f4a1806c420 2023-10-30T12:45:27.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f4a167fc640 1 -- 172.21.15.8:0/1423258953 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000aab80 con 0x7f4a1806c420 2023-10-30T12:45:27.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/831716468 >> v1:172.21.15.46:6789/0 conn(0x7f2a9c06ee90 legacy=0x7f2a9c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/831716468 >> v1:172.21.15.8:6789/0 conn(0x7f2a9c151850 legacy=0x7f2a9c0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb56c0b7040 con 0x7fb5780dd430 2023-10-30T12:45:27.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb95f7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 0x7fb9600ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36204/0 (socket says 172.21.15.8:36204) 2023-10-30T12:45:27.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb95f7fe640 1 -- 172.21.15.8:0/3258425495 learned_addr learned my addr 172.21.15.8:0/3258425495 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7fb95effd640 1 --2- 172.21.15.8:0/3258425495 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 0x7fb9600c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2e75160640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e7006fdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0097c0 tx=0x7f2e6c01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4485478802695a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.993+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/831716468 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c06ddb0 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb57d7fa640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb5640664a0 0x7fb564068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb95effd640 1 -- 172.21.15.8:0/3258425495 >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 msgr2=0x7fb9600ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb95effd640 1 --2- 172.21.15.8:0/3258425495 >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 0x7fb9600ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb95effd640 1 -- 172.21.15.8:0/3258425495 >> v1:172.21.15.46:6789/0 conn(0x7fb9600c46e0 legacy=0x7fb9600cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb56c1222f0 con 0x7fb5780dd430 2023-10-30T12:45:27.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1423258953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 msgr2=0x7f4a180c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1423258953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c57a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a1815b7a0 tx=0x7f4a0001c4f0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f2e7495f640 1 -- 172.21.15.8:0/1336393415 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c0a88f0 con 0x7f2e700c7110 2023-10-30T12:45:27.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1423258953 shutdown_connections 2023-10-30T12:45:27.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1423258953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c57a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1423258953 >> v2:172.21.15.8:3300/0 conn(0x7f4a1806b760 0x7f4a1815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1423258953 >> 172.21.15.8:0/1423258953 conn(0x7f4a180cac70 msgr2=0x7f4a180cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f2e7495f640 1 -- 172.21.15.8:0/1336393415 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c0a8a50 con 0x7f2e700c7110 2023-10-30T12:45:27.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f2e7495f640 1 -- 172.21.15.8:0/1336393415 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c0a8de0 con 0x7f2e700c7110 2023-10-30T12:45:27.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9580047d0 0x7fe958004c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe964001690 tx=0x7fe96401c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f73225a61dd95c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1423258953 shutdown_connections 2023-10-30T12:45:27.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67b0071a90 con 0x7f67b006b760 2023-10-30T12:45:27.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb95e7fc640 1 -- 172.21.15.8:0/3258425495 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255379785 0 0) 0x7fb96015e090 con 0x7fb9600c46e0 2023-10-30T12:45:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb95effd640 1 -- 172.21.15.8:0/3258425495 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960161db0 con 0x7fb9601583e0 2023-10-30T12:45:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640a4ea0 con 0x7fe9580047d0 2023-10-30T12:45:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb57d7fa640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7fb56406e6c0 0x7fb564070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb564071190 con 0x7fb56406e6c0 2023-10-30T12:45:27.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb56c1226a0 con 0x7fb5780dd430 2023-10-30T12:45:27.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9640a7070 con 0x7fe9580047d0 2023-10-30T12:45:27.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe96dffb640 1 -- 172.21.15.8:0/1814770627 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640074d0 con 0x7fe9580047d0 2023-10-30T12:45:27.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== osd.1 v2:172.21.15.8:6804/2525970788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7758071260 con 0x7f775806e790 2023-10-30T12:45:27.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb584b01640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb5640664a0 0x7fb564068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb585302640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7fb56406e6c0 0x7fb564070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9580047d0 msgr2=0x7fe958004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe97593e640 1 --2- 172.21.15.8:0/1814770627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9580047d0 0x7fe958004c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe964001690 tx=0x7fe96401c2f0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67ac00f3e0 con 0x7f67b006b760 2023-10-30T12:45:27.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe96f7fe640 1 -- 172.21.15.8:0/1814770627 reap_dead start 2023-10-30T12:45:27.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 shutdown_connections 2023-10-30T12:45:27.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 >> 172.21.15.8:0/1814770627 conn(0x7fe9680502b0 msgr2=0x7fe9680506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f2e77bc5640 1 -- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 msgr2=0x7f2e7006fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e7006fdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0097c0 tx=0x7f2e6c01ec30 comp rx=0 tx=0).stop 2023-10-30T12:45:27.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67a67fc640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67940664a0 0x7f6794068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67ac083eb0 con 0x7f67b006b760 2023-10-30T12:45:27.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67b516e640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67940664a0 0x7f6794068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67a67fc640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7f679406e4d0 0x7f67940708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6794070fa0 con 0x7f679406e4d0 2023-10-30T12:45:27.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1423258953 wait complete. 2023-10-30T12:45:27.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:26.997+0000 7fb585302640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7fb56406e6c0 0x7fb564070ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67ac051350 con 0x7f67b006b760 2023-10-30T12:45:27.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb584b01640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb5640664a0 0x7fb564068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb574004fa0 tx=0x7fb574007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 shutdown_connections 2023-10-30T12:45:27.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2a9b7fe640 1 --2- 172.21.15.8:0/831716468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c0c23c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009d90 tx=0x7f2a9401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ce28cbc478934c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fe97593e640 1 -- 172.21.15.8:0/1814770627 wait complete. 2023-10-30T12:45:27.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 Processor -- start 2023-10-30T12:45:27.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f67b516e640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67940664a0 0x7f6794068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67b01541c0 tx=0x7f67a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f67b6170640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7f679406e4d0 0x7f67940708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== osd.0 v2:172.21.15.8:6801/2064691129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fb564071190 con 0x7fb56406e6c0 2023-10-30T12:45:27.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 -- start start 2023-10-30T12:45:27.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2a99ffb640 1 -- 172.21.15.8:0/831716468 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940a5810 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2a99ffb640 1 -- 172.21.15.8:0/831716468 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940a5970 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2a99ffb640 1 -- 172.21.15.8:0/831716468 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940a5d00 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 -- 172.21.15.8:0/1336393415 shutdown_connections 2023-10-30T12:45:27.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e7014f910 0x7f2e7006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/1336393415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e7006fdf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 -- 172.21.15.8:0/1336393415 >> 172.21.15.8:0/1336393415 conn(0x7f2e700cac70 msgr2=0x7f2e700cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f67b6170640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7f679406e4d0 0x7f67940708f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 -- 172.21.15.8:0/831716468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c0c1f90 msgr2=0x7f2a9c0c23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 --2- 172.21.15.8:0/831716468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c0c23c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009d90 tx=0x7f2a9401c950 comp rx=0 tx=0).stop 2023-10-30T12:45:27.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 -- 172.21.15.8:0/1336393415 shutdown_connections 2023-10-30T12:45:27.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a1806b760 0x7f4a181591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4a1806bb40 0x7f4a18159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180caae0 con 0x7f4a1806b760 2023-10-30T12:45:27.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c2ad0 con 0x7f4a1806c420 2023-10-30T12:45:27.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a1e09a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c93f0 con 0x7f4a1806bb40 2023-10-30T12:45:27.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 -- 172.21.15.8:0/831716468 shutdown_connections 2023-10-30T12:45:27.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 --2- 172.21.15.8:0/831716468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c0c23c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 -- 172.21.15.8:0/831716468 >> 172.21.15.8:0/831716468 conn(0x7f2a9c06fbe0 msgr2=0x7f2a9c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== osd.2 v2:172.21.15.8:6803/713612811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f6794070fa0 con 0x7f679406e4d0 2023-10-30T12:45:27.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a17fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49968/0 (socket says 172.21.15.8:49968) 2023-10-30T12:45:27.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a17fff640 1 -- 172.21.15.8:0/1034474413 learned_addr learned my addr 172.21.15.8:0/1034474413 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f4a16ffd640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4a1806bb40 0x7f4a18159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb95effd640 1 --2- 172.21.15.8:0/3258425495 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 0x7fb9600c41a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb958009d90 tx=0x7fb95801f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc921d560742ef99 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb95e7fc640 1 -- 172.21.15.8:0/3258425495 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9580af600 con 0x7fb9601583e0 2023-10-30T12:45:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb95e7fc640 1 -- 172.21.15.8:0/3258425495 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9580ab050 con 0x7fb9601583e0 2023-10-30T12:45:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7fb95e7fc640 1 -- 172.21.15.8:0/3258425495 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9580af980 con 0x7fb9601583e0 2023-10-30T12:45:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a1806b760 0x7f4a181591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 Processor -- start 2023-10-30T12:45:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 -- 172.21.15.8:0/831716468 shutdown_connections 2023-10-30T12:45:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2aa1980640 1 -- 172.21.15.8:0/831716468 wait complete. 2023-10-30T12:45:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 -- start start 2023-10-30T12:45:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.001+0000 7f2e77bc5640 1 -- 172.21.15.8:0/1336393415 wait complete. 2023-10-30T12:45:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f2e77bc5640 1 Processor -- start 2023-10-30T12:45:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f2e77bc5640 1 -- start start 2023-10-30T12:45:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f2aa1980640 1 Processor -- start 2023-10-30T12:45:27.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9580047d0 0x7fe968173ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9680dd200 0x7fe968045560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 0x7fe9680c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680ce660 con 0x7fe9580047d0 2023-10-30T12:45:27.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968173810 con 0x7fe9680cd160 2023-10-30T12:45:27.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe97593e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680db9b0 con 0x7fe9680dd200 2023-10-30T12:45:27.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f2aa1980640 1 -- start start 2023-10-30T12:45:27.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 -- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4a1806bb40 msgr2=0x7f4a18159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4a1806bb40 0x7f4a18159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 -- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 msgr2=0x7f4a180c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1806c420 0x7f4a180c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000094d0 con 0x7f4a1806b760 2023-10-30T12:45:27.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7f4a177fe640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a1806b760 0x7f4a181591e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000b3f0 tx=0x7f4a1000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fb965ad6640 1 -- 172.21.15.8:0/3258425495 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 msgr2=0x7fb9600c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 0x7fe9680c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe96e7fc640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9680dd200 0x7fe968045560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 0x7fe9680c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49974/0 (socket says 172.21.15.8:49974) 2023-10-30T12:45:27.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fe96f7fe640 1 -- 172.21.15.8:0/3797533311 learned_addr learned my addr 172.21.15.8:0/3797533311 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.005+0000 7fb965ad6640 1 --2- 172.21.15.8:0/3258425495 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 0x7fb9600c41a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb958009d90 tx=0x7fb95801f130 comp rx=0 tx=0).stop 2023-10-30T12:45:27.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c06ee90 0x7f2a9c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96effd640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9580047d0 0x7fe968173ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 -- 172.21.15.8:0/3258425495 shutdown_connections 2023-10-30T12:45:27.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 0x7f2a9c0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c1f2bc0 con 0x7f2a9c0c1f90 2023-10-30T12:45:27.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0cb110 con 0x7f2a9c06ee90 2023-10-30T12:45:27.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2aa1980640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0cdc40 con 0x7f2a9c151850 2023-10-30T12:45:27.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 -- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 msgr2=0x7fe9680c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 0x7fe9680c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 -- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9580047d0 msgr2=0x7fe968173ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9580047d0 0x7fe968173ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680db6d0 con 0x7fe9680dd200 2023-10-30T12:45:27.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96f7fe640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9680cd160 0x7fe9680c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:27.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe96e7fc640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9680dd200 0x7fe968045560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a0c0 tx=0x7fe958005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964007290 con 0x7fe9680dd200 2023-10-30T12:45:27.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9640b2040 con 0x7fe9680dd200 2023-10-30T12:45:27.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10011830 con 0x7f4a1806b760 2023-10-30T12:45:27.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10011990 con 0x7f4a1806b760 2023-10-30T12:45:27.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 --2- 172.21.15.8:0/3258425495 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9601583e0 0x7fb9600c41a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 --2- 172.21.15.8:0/3258425495 >> v2:172.21.15.8:3300/0 conn(0x7fb960158000 0x7fb9600ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 -- 172.21.15.8:0/3258425495 >> 172.21.15.8:0/3258425495 conn(0x7fb9600715c0 msgr2=0x7fb9600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 -- 172.21.15.8:0/3258425495 shutdown_connections 2023-10-30T12:45:27.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fb965ad6640 1 -- 172.21.15.8:0/3258425495 wait complete. 2023-10-30T12:45:27.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 0x7f2a9c0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c06ee90 0x7f2a9c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 0x7f2a9c0c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33378/0 (socket says 172.21.15.8:33378) 2023-10-30T12:45:27.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c06ee90 0x7f2a9c0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:49990/0 (socket says 172.21.15.8:49990) 2023-10-30T12:45:27.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/524894644 learned_addr learned my addr 172.21.15.8:0/524894644 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9a7fc640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c06ee90 msgr2=0x7f2a9c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c06ee90 0x7f2a9c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a9c0c1f90 msgr2=0x7f2a9c144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a9c0c1f90 0x7f2a9c144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009a70 con 0x7f2a9c151850 2023-10-30T12:45:27.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e70138d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e700cb790 0x7f2e701554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e7014f910 0x7f2e70155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e70138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e70138d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50006/0 (socket says 172.21.15.8:50006) 2023-10-30T12:45:27.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 -- 172.21.15.8:0/4176694583 learned_addr learned my addr 172.21.15.8:0/4176694583 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75160640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e700cb790 0x7f2e701554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701525b0 con 0x7f2e7014f910 2023-10-30T12:45:27.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70150a40 con 0x7f2e700c7110 2023-10-30T12:45:27.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701392b0 con 0x7f2e700cb790 2023-10-30T12:45:27.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964007290 con 0x7fe9680dd200 2023-10-30T12:45:27.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10011d40 con 0x7f4a1806b760 2023-10-30T12:45:27.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9680da290 con 0x7fe9680dd200 2023-10-30T12:45:27.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681741e0 con 0x7fe9680dd200 2023-10-30T12:45:27.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180ca800 con 0x7f4a1806b760 2023-10-30T12:45:27.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180c2ce0 con 0x7f4a1806b760 2023-10-30T12:45:27.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a180ca3e0 con 0x7f4a1806b760 2023-10-30T12:45:27.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7fb965ad6640 1 Processor -- start 2023-10-30T12:45:27.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2a9b7fe640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 0x7f2a9c0c59d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009d90 tx=0x7f2a940196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940ad850 con 0x7f2a9c151850 2023-10-30T12:45:27.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940ad9b0 con 0x7f2a9c151850 2023-10-30T12:45:27.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940add60 con 0x7f2a9c151850 2023-10-30T12:45:27.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c06ba70 con 0x7f2a9c151850 2023-10-30T12:45:27.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c1f28e0 con 0x7f2a9c151850 2023-10-30T12:45:27.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 -- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e700cb790 msgr2=0x7f2e701554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2e700cb790 0x7f2e701554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 -- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e7014f910 msgr2=0x7f2e70155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e7014f910 0x7f2e70155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c009420 con 0x7f2e700c7110 2023-10-30T12:45:27.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.009+0000 7f2e75961640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e70138d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c4a0 tx=0x7f2e6000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60005cf0 con 0x7f2e700c7110 2023-10-30T12:45:27.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e60005e70 con 0x7f2e700c7110 2023-10-30T12:45:27.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e600072f0 con 0x7f2e700c7110 2023-10-30T12:45:27.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e70152800 con 0x7f2e700c7110 2023-10-30T12:45:27.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e70150c90 con 0x7f2e700c7110 2023-10-30T12:45:27.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe968173670 con 0x7fe9680dd200 2023-10-30T12:45:27.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e70156670 con 0x7f2e700c7110 2023-10-30T12:45:27.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a9c06bf00 con 0x7f2a9c151850 2023-10-30T12:45:27.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7fb965ad6640 1 -- start start 2023-10-30T12:45:27.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.013+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9640073f0 con 0x7fe9680dd200 2023-10-30T12:45:27.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 0x7fb96015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb960158000 0x7fb9601f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2e60007450 con 0x7f2e700c7110 2023-10-30T12:45:27.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 0x7fb96015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 0x7fb96015efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33390/0 (socket says 172.21.15.8:33390) 2023-10-30T12:45:27.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 -- 172.21.15.8:0/1192195638 learned_addr learned my addr 172.21.15.8:0/1192195638 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2a940b4070 con 0x7f2a9c151850 2023-10-30T12:45:27.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9601583e0 0x7fb9601f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960152680 con 0x7fb960158000 2023-10-30T12:45:27.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960152920 con 0x7fb9601583e0 2023-10-30T12:45:27.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb960152c00 con 0x7fb9600c46e0 2023-10-30T12:45:27.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95ffff640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9601583e0 0x7fb9601f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe957fff640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe95c066590 0x7fe95c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe96411cb70 con 0x7fe9680dd200 2023-10-30T12:45:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe957fff640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7fe95c06e730 0x7fe95c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe95c071200 con 0x7fe95c06e730 2023-10-30T12:45:27.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 -- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9601583e0 msgr2=0x7fb9601f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9601583e0 0x7fb9601f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 -- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb960158000 msgr2=0x7fb9601f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb960158000 0x7fb9601f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958009a70 con 0x7fb9600c46e0 2023-10-30T12:45:27.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e667fc640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2e540664a0 0x7f2e54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e600833f0 con 0x7f2e700c7110 2023-10-30T12:45:27.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe96f7fe640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7fe95c06e730 0x7fe95c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2a7c0664a0 0x7f2a7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e75160640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2e540664a0 0x7f2e54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e667fc640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f2e5406e640 0x7f2e54070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e54071110 con 0x7f2e5406e640 2023-10-30T12:45:27.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb95f7fe640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 0x7fb96015efe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e7e0 tx=0x7fb95400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a9411f790 con 0x7f2a9c151850 2023-10-30T12:45:27.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7f2a7c06e640 0x7f2a7c070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a7c071110 con 0x7f2a7c06e640 2023-10-30T12:45:27.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9411fbb0 con 0x7f2a9c151850 2023-10-30T12:45:27.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f2e76162640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f2e5406e640 0x7f2e54070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fe96effd640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe95c066590 0x7fe95c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400c8d0 con 0x7fb9600c46e0 2023-10-30T12:45:27.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a1001a400 con 0x7f4a1806b760 2023-10-30T12:45:27.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f49fc0664d0 0x7f49fc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a10084530 con 0x7f4a1806b760 2023-10-30T12:45:27.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f49fc06e670 0x7f49fc070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49fc071140 con 0x7f49fc06e670 2023-10-30T12:45:27.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.017+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a100848e0 con 0x7f4a1806b760 2023-10-30T12:45:27.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f4a17fff640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f49fc06e670 0x7f49fc070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95400ee90 con 0x7fb9600c46e0 2023-10-30T12:45:27.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9601f63c0 con 0x7fb9600c46e0 2023-10-30T12:45:27.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400cc50 con 0x7fb9600c46e0 2023-10-30T12:45:27.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2a9a7fc640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7f2a7c06e640 0x7f2a7c070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2e75160640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2e540664a0 0x7f2e54068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b2000 tx=0x7f2e6c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2e76162640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f2e5406e640 0x7f2e54070a60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9601f6980 con 0x7fb9600c46e0 2023-10-30T12:45:27.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f4a16ffd640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f49fc0664d0 0x7f49fc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2a9affd640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2a7c0664a0 0x7f2a7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f4a17fff640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f49fc06e670 0x7f49fc070a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fe96effd640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe95c066590 0x7fe95c068a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe970069800 tx=0x7fe97006d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fe96f7fe640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7fe95c06e730 0x7fe95c070b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2a9affd640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2a7c0664a0 0x7f2a7c068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a84005890 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb960162080 con 0x7fb9600c46e0 2023-10-30T12:45:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2a9a7fc640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7f2a7c06e640 0x7f2a7c070a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== osd.7 v2:172.21.15.46:6808/1029552049 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2e54071110 con 0x7f2e5406e640 2023-10-30T12:45:27.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.021+0000 7f4a16ffd640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f49fc0664d0 0x7f49fc068990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a180c6c30 tx=0x7f4a08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== osd.3 v2:172.21.15.8:6800/469140225 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe95c071200 con 0x7fe95c06e730 2023-10-30T12:45:27.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== osd.4 v2:172.21.15.46:6810/4276465018 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f49fc071140 con 0x7f49fc06e670 2023-10-30T12:45:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 --> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb578006c50 con 0x7fb56406e6c0 2023-10-30T12:45:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== osd.6 v2:172.21.15.46:6824/815955451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2a7c071110 con 0x7f2a7c06e640 2023-10-30T12:45:27.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb57d7fa640 1 -- 172.21.15.8:0/3892925147 <== osd.0 v2:172.21.15.8:6801/2064691129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb58004e6e0 con 0x7fb56406e6c0 2023-10-30T12:45:27.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe964007290 con 0x7fe9680dd200 2023-10-30T12:45:27.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7fb56406e6c0 msgr2=0x7fb564070ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7fb56406e6c0 0x7fb564070ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb5640664a0 msgr2=0x7fb564068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb5640664a0 0x7fb564068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb574004fa0 tx=0x7fb574007ab0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 msgr2=0x7fb5780354e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 --2- 172.21.15.8:0/3892925147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5780dd430 0x7fb5780354e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb56c01c860 tx=0x7fb56c01b9d0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb585302640 1 -- 172.21.15.8:0/3892925147 reap_dead start 2023-10-30T12:45:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 shutdown_connections 2023-10-30T12:45:27.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.025+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 >> 172.21.15.8:0/3892925147 conn(0x7fb578050170 msgr2=0x7fb5780e6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 shutdown_connections 2023-10-30T12:45:27.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7fb586d65640 1 -- 172.21.15.8:0/3892925147 wait complete. 2023-10-30T12:45:27.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 --> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f776c006c50 con 0x7f775806e790 2023-10-30T12:45:27.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb954013410 con 0x7fb9600c46e0 2023-10-30T12:45:27.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e60004b70 con 0x7f2e700c7110 2023-10-30T12:45:27.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.029+0000 7f77717fa640 1 -- 172.21.15.8:0/848749622 <== osd.1 v2:172.21.15.8:6804/2525970788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7768009e90 con 0x7f775806e790 2023-10-30T12:45:27.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7fb947fff640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb94c066570 0x7fb94c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb954084320 con 0x7fb9600c46e0 2023-10-30T12:45:27.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7fb947fff640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fb94c06e790 0x7fb94c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb94c071260 con 0x7fb94c06e790 2023-10-30T12:45:27.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9540846d0 con 0x7fb9600c46e0 2023-10-30T12:45:27.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f775806e790 msgr2=0x7f7758070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f775806e790 0x7f7758070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7758066570 msgr2=0x7f7758068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7758066570 0x7f7758068a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77740732e0 tx=0x7f7774075690 comp rx=0 tx=0).stop 2023-10-30T12:45:27.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0e6760 msgr2=0x7f776c0e59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 --2- 172.21.15.8:0/848749622 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f776c0e6760 0x7f776c0e59f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f77600af000 tx=0x7f776001bab0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f77793e3640 1 -- 172.21.15.8:0/848749622 reap_dead start 2023-10-30T12:45:27.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 shutdown_connections 2023-10-30T12:45:27.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 >> 172.21.15.8:0/848749622 conn(0x7f776c050170 msgr2=0x7f776c005d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 shutdown_connections 2023-10-30T12:45:27.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.033+0000 7f777ae46640 1 -- 172.21.15.8:0/848749622 wait complete. 2023-10-30T12:45:27.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7fb95effd640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb94c066570 0x7fb94c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7fb95ffff640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fb94c06e790 0x7fb94c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7fb95effd640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb94c066570 0x7fb94c068a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b1000 tx=0x7fb958004ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 --> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67b00717b0 con 0x7f679406e4d0 2023-10-30T12:45:27.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7fb95ffff640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fb94c06e790 0x7fb94c070bb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7f67a67fc640 1 -- 172.21.15.8:0/3190729618 <== osd.2 v2:172.21.15.8:6803/713612811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67a80ad820 con 0x7f679406e4d0 2023-10-30T12:45:27.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7f679406e4d0 msgr2=0x7f67940708f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.037+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7f679406e4d0 0x7f67940708f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.041+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== osd.5 v2:172.21.15.46:6800/3519076799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fb94c071260 con 0x7fb94c06e790 2023-10-30T12:45:27.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67940664a0 msgr2=0x7f6794068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67940664a0 0x7f6794068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67b01541c0 tx=0x7f67a0009040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 msgr2=0x7f67b015f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 --2- 172.21.15.8:0/3190729618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b006b760 0x7f67b015f6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b5b0 tx=0x7f67ac00ba70 comp rx=0 tx=0).stop 2023-10-30T12:45:27.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b6170640 1 -- 172.21.15.8:0/3190729618 reap_dead start 2023-10-30T12:45:27.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 shutdown_connections 2023-10-30T12:45:27.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 >> 172.21.15.8:0/3190729618 conn(0x7f67b00cac70 msgr2=0x7f67b0154710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 shutdown_connections 2023-10-30T12:45:27.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f67b7bd3640 1 -- 172.21.15.8:0/3190729618 wait complete. 2023-10-30T12:45:27.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.045+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 --> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e700c5c50 con 0x7f2e5406e640 2023-10-30T12:45:27.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 --> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a18071a90 con 0x7f49fc06e670 2023-10-30T12:45:27.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e667fc640 1 -- 172.21.15.8:0/4176694583 <== osd.7 v2:172.21.15.46:6808/1029552049 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e6800f040 con 0x7f2e5406e640 2023-10-30T12:45:27.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f2e5406e640 msgr2=0x7f2e54070a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f2e5406e640 0x7f2e54070a60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2e540664a0 msgr2=0x7f2e54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2e540664a0 0x7f2e54068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b2000 tx=0x7f2e6c008040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 msgr2=0x7f2e70138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 --2- 172.21.15.8:0/4176694583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e700c7110 0x7f2e70138d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c4a0 tx=0x7f2e6000c960 comp rx=0 tx=0).stop 2023-10-30T12:45:27.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e76162640 1 -- 172.21.15.8:0/4176694583 reap_dead start 2023-10-30T12:45:27.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 shutdown_connections 2023-10-30T12:45:27.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 >> 172.21.15.8:0/4176694583 conn(0x7f2e700cac70 msgr2=0x7f2e7014ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a0ffff640 1 -- 172.21.15.8:0/1034474413 <== osd.4 v2:172.21.15.46:6810/4276465018 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a000aa920 con 0x7f49fc06e670 2023-10-30T12:45:27.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 shutdown_connections 2023-10-30T12:45:27.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f2e77bc5640 1 -- 172.21.15.8:0/4176694583 wait complete. 2023-10-30T12:45:27.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f49fc06e670 msgr2=0x7f49fc070a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f49fc06e670 0x7f49fc070a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f49fc0664d0 msgr2=0x7f49fc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f49fc0664d0 0x7f49fc068990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a180c6c30 tx=0x7f4a08009040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a1806b760 msgr2=0x7f4a181591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 --2- 172.21.15.8:0/1034474413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a1806b760 0x7f4a181591e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000b3f0 tx=0x7f4a1000b8b0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a17fff640 1 -- 172.21.15.8:0/1034474413 reap_dead start 2023-10-30T12:45:27.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 shutdown_connections 2023-10-30T12:45:27.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.049+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 >> 172.21.15.8:0/1034474413 conn(0x7f4a180cac70 msgr2=0x7f4a18153be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.057+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 shutdown_connections 2023-10-30T12:45:27.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.057+0000 7f4a1e09a640 1 -- 172.21.15.8:0/1034474413 wait complete. 2023-10-30T12:45:27.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.057+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 --> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9680dbe50 con 0x7fe95c06e730 2023-10-30T12:45:27.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.057+0000 7fe957fff640 1 -- 172.21.15.8:0/3797533311 <== osd.3 v2:172.21.15.8:6800/469140225 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe96000b810 con 0x7fe95c06e730 2023-10-30T12:45:27.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7fe95c06e730 msgr2=0x7fe95c070b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7fe95c06e730 0x7fe95c070b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 --> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a9c06e930 con 0x7f2a7c06e640 2023-10-30T12:45:27.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 --> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb960162360 con 0x7fb94c06e790 2023-10-30T12:45:27.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe95c066590 msgr2=0x7fe95c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe95c066590 0x7fe95c068a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe970069800 tx=0x7fe97006d040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9680dd200 msgr2=0x7fe968045560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 --2- 172.21.15.8:0/3797533311 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe9680dd200 0x7fe968045560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a0c0 tx=0x7fe958005800 comp rx=0 tx=0).stop 2023-10-30T12:45:27.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe96f7fe640 1 -- 172.21.15.8:0/3797533311 reap_dead start 2023-10-30T12:45:27.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 shutdown_connections 2023-10-30T12:45:27.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.061+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 >> 172.21.15.8:0/3797533311 conn(0x7fe9680502b0 msgr2=0x7fe9680d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7f2a93fff640 1 -- 172.21.15.8:0/524894644 <== osd.6 v2:172.21.15.46:6824/815955451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a8c001450 con 0x7f2a7c06e640 2023-10-30T12:45:27.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 shutdown_connections 2023-10-30T12:45:27.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fe97593e640 1 -- 172.21.15.8:0/3797533311 wait complete. 2023-10-30T12:45:27.157 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.158 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T12:45:27.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb947fff640 1 -- 172.21.15.8:0/1192195638 <== osd.5 v2:172.21.15.46:6800/3519076799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9500012c0 con 0x7fb94c06e790 2023-10-30T12:45:27.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fb94c06e790 msgr2=0x7fb94c070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fb94c06e790 0x7fb94c070bb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb94c066570 msgr2=0x7fb94c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb94c066570 0x7fb94c068a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b1000 tx=0x7fb958004ac0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 msgr2=0x7fb96015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 --2- 172.21.15.8:0/1192195638 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb9600c46e0 0x7fb96015efe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e7e0 tx=0x7fb95400eca0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7f2a7c06e640 msgr2=0x7f2a7c070a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7f2aa1980640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7f2a7c06e640 0x7f2a7c070a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb95ffff640 1 -- 172.21.15.8:0/1192195638 reap_dead start 2023-10-30T12:45:27.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 shutdown_connections 2023-10-30T12:45:27.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.065+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 >> 172.21.15.8:0/1192195638 conn(0x7fb9600715c0 msgr2=0x7fb9600c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.163 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.163 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T12:45:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2a7c0664a0 msgr2=0x7f2a7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2a7c0664a0 0x7f2a7c068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a84005890 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 msgr2=0x7f2a9c0c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 --2- 172.21.15.8:0/524894644 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2a9c151850 0x7f2a9c0c59d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009d90 tx=0x7f2a940196f0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2a9b7fe640 1 -- 172.21.15.8:0/524894644 reap_dead start 2023-10-30T12:45:27.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 shutdown_connections 2023-10-30T12:45:27.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7fb965ad6640 1 -- 172.21.15.8:0/1192195638 wait complete. 2023-10-30T12:45:27.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 shutdown_connections 2023-10-30T12:45:27.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 >> 172.21.15.8:0/524894644 conn(0x7f2a9c06fbe0 msgr2=0x7f2a9c0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 shutdown_connections 2023-10-30T12:45:27.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.069+0000 7f2aa1980640 1 -- 172.21.15.8:0/524894644 wait complete. 2023-10-30T12:45:27.167 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.168 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T12:45:27.169 INFO:teuthology.orchestra.run.smithi008.stdout:55834574856 2023-10-30T12:45:27.169 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T12:45:27.170 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.170 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T12:45:27.171 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.172 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T12:45:27.173 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.173 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T12:45:27.174 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.175 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T12:45:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 Processor -- start 2023-10-30T12:45:27.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 -- start start 2023-10-30T12:45:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2c0c0d6e40 0x7f2c0c0dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c047e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c04e130 con 0x7f2c0c0e0100 2023-10-30T12:45:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c04cec0 con 0x7f2c0c0d6e40 2023-10-30T12:45:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.445+0000 7f2c19d80640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c009a40 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c13fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c0c0e0100 0x7f2c0c04d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:33012/0 (socket says 172.21.15.8:33012) 2023-10-30T12:45:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c13fff640 1 -- 172.21.15.8:0/1183096504 learned_addr learned my addr 172.21.15.8:0/1183096504 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c12ffd640 1 --2- 172.21.15.8:0/1183096504 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c047e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c137fe640 1 --2- 172.21.15.8:0/1183096504 >> v2:172.21.15.8:3300/0 conn(0x7f2c0c0d6e40 0x7f2c0c0dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c12ffd640 1 -- 172.21.15.8:0/1183096504 >> v2:172.21.15.8:3300/0 conn(0x7f2c0c0d6e40 msgr2=0x7f2c0c0dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c12ffd640 1 --2- 172.21.15.8:0/1183096504 >> v2:172.21.15.8:3300/0 conn(0x7f2c0c0d6e40 0x7f2c0c0dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c12ffd640 1 -- 172.21.15.8:0/1183096504 >> v1:172.21.15.46:6789/0 conn(0x7f2c0c0e0100 legacy=0x7f2c0c04d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.449+0000 7f2c12ffd640 1 -- 172.21.15.8:0/1183096504 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0d7220 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.453+0000 7f2c12ffd640 1 --2- 172.21.15.8:0/1183096504 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c047e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097c0 tx=0x7f2c0001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b06ba9ca1715a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c127fc640 1 -- 172.21.15.8:0/1183096504 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000ab600 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c127fc640 1 -- 172.21.15.8:0/1183096504 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c000a7050 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c127fc640 1 -- 172.21.15.8:0/1183096504 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000ab980 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 -- 172.21.15.8:0/1183096504 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 msgr2=0x7f2c0c047e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 --2- 172.21.15.8:0/1183096504 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c047e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097c0 tx=0x7f2c0001c440 comp rx=0 tx=0).stop 2023-10-30T12:45:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 -- 172.21.15.8:0/1183096504 shutdown_connections 2023-10-30T12:45:27.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 --2- 172.21.15.8:0/1183096504 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c047e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 --2- 172.21.15.8:0/1183096504 >> v2:172.21.15.8:3300/0 conn(0x7f2c0c0d6e40 0x7f2c0c0dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 -- 172.21.15.8:0/1183096504 >> 172.21.15.8:0/1183096504 conn(0x7f2c0c00a240 msgr2=0x7f2c0c00e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 -- 172.21.15.8:0/1183096504 shutdown_connections 2023-10-30T12:45:27.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.457+0000 7f2c19d80640 1 -- 172.21.15.8:0/1183096504 wait complete. 2023-10-30T12:45:27.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 Processor -- start 2023-10-30T12:45:27.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 -- start start 2023-10-30T12:45:27.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 Processor -- start 2023-10-30T12:45:27.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 Processor -- start 2023-10-30T12:45:27.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe6300d6dd0 0x7fe6300dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe63004e080 0x7fe63004e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6300d71b0 con 0x7fe630047540 2023-10-30T12:45:27.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6300e0d40 con 0x7fe6300d6dd0 2023-10-30T12:45:27.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe63759e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6300dda80 con 0x7fe63004e080 2023-10-30T12:45:27.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 -- start start 2023-10-30T12:45:27.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 -- start start 2023-10-30T12:45:27.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c0d5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c0c0d6e40 0x7f2c0c035860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c1aa0 con 0x7f2c0c0d6e40 2023-10-30T12:45:27.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0ded70 con 0x7f2c0c0e0100 2023-10-30T12:45:27.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c19d80640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c17ce40 con 0x7f2c0c00ebe0 2023-10-30T12:45:27.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ecc00eb70 0x7f8ecc0075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc04eef0 con 0x7f8ecc050950 2023-10-30T12:45:27.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0df920 con 0x7f8ecc00eb70 2023-10-30T12:45:27.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8eda0dc640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0dfc00 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe635d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe63004e080 0x7fe63004e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c13fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c13fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50034/0 (socket says 172.21.15.8:50034) 2023-10-30T12:45:27.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c13fff640 1 -- 172.21.15.8:0/3348153542 learned_addr learned my addr 172.21.15.8:0/3348153542 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f2c137fe640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c0d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7ffbf613e640 1 Processor -- start 2023-10-30T12:45:27.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8ed90da640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8ed90da640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33398/0 (socket says 172.21.15.8:33398) 2023-10-30T12:45:27.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8ed90da640 1 -- 172.21.15.8:0/1966924003 learned_addr learned my addr 172.21.15.8:0/1966924003 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f8ed88d9640 1 --2- 172.21.15.8:0/1966924003 >> v2:172.21.15.46:3300/0 conn(0x7f8ecc00eb70 0x7f8ecc0075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe635d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe63004e080 0x7fe63004e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50032/0 (socket says 172.21.15.8:50032) 2023-10-30T12:45:27.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7fe635d9b640 1 -- 172.21.15.8:0/2996913430 learned_addr learned my addr 172.21.15.8:0/2996913430 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.461+0000 7f56b5e09640 1 Processor -- start 2023-10-30T12:45:27.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628607529 0 0) 0x7fe6300d71b0 con 0x7fe630047540 2023-10-30T12:45:27.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe620003530 con 0x7fe630047540 2023-10-30T12:45:27.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 -- start start 2023-10-30T12:45:27.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f8ed90da640 1 -- 172.21.15.8:0/1966924003 >> v2:172.21.15.46:3300/0 conn(0x7f8ecc00eb70 msgr2=0x7f8ecc0075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f8ed90da640 1 --2- 172.21.15.8:0/1966924003 >> v2:172.21.15.46:3300/0 conn(0x7f8ecc00eb70 0x7f8ecc0075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f8ed90da640 1 -- 172.21.15.8:0/1966924003 >> v1:172.21.15.8:6789/0 conn(0x7f8ecc050950 legacy=0x7f8ecc04f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f8ed90da640 1 -- 172.21.15.8:0/1966924003 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0497d0 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 -- start start 2023-10-30T12:45:27.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3547884869 0 0) 0x7fe620003530 con 0x7fe630047540 2023-10-30T12:45:27.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 >> v2:172.21.15.46:3300/0 conn(0x7fe63004e080 msgr2=0x7fe63004e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 --2- 172.21.15.8:0/2996913430 >> v2:172.21.15.46:3300/0 conn(0x7fe63004e080 0x7fe63004e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 >> v2:172.21.15.8:3300/0 conn(0x7fe6300d6dd0 msgr2=0x7fe6300dd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 --2- 172.21.15.8:0/2996913430 >> v2:172.21.15.8:3300/0 conn(0x7fe6300d6dd0 0x7fe6300dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe630009af0 con 0x7fe630047540 2023-10-30T12:45:27.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fe62c001680 con 0x7fe630047540 2023-10-30T12:45:27.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 >> v1:172.21.15.8:6789/0 conn(0x7fe630047540 legacy=0x7fe630047970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbf00c6320 0x7ffbf006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe620003980 0x7fe620003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe620004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe620005190 0x7fe6200141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300e0d40 con 0x7fe6200047d0 2023-10-30T12:45:27.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200147f0 con 0x7fe620003980 2023-10-30T12:45:27.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ffbf014fc50 0x7ffbf0070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbef7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbf00c6320 0x7ffbf006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbef7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbf00c6320 0x7ffbf006fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50050/0 (socket says 172.21.15.8:50050) 2023-10-30T12:45:27.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200149b0 con 0x7fe620005190 2023-10-30T12:45:27.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbef7fe640 1 -- 172.21.15.8:0/435563460 learned_addr learned my addr 172.21.15.8:0/435563460 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe620004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7fe636d9d640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe620005190 0x7fe6200141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbf00c55f0 con 0x7ffbf0155c40 2023-10-30T12:45:27.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbeeffd640 1 --2- 172.21.15.8:0/435563460 >> v2:172.21.15.8:3300/0 conn(0x7ffbf014fc50 0x7ffbf0070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbf00c5f00 con 0x7ffbf014fc50 2023-10-30T12:45:27.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbf0071350 con 0x7ffbf00c6320 2023-10-30T12:45:27.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 0x7f56b006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f56b014f910 0x7f56b006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b01563b0 con 0x7f56b00cb790 2023-10-30T12:45:27.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b00c6030 con 0x7f56b014f910 2023-10-30T12:45:27.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56b5e09640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b00c5690 con 0x7f56b00c7110 2023-10-30T12:45:27.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 -- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 msgr2=0x7f2c0c0d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c0d5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 -- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c0c0d6e40 msgr2=0x7f2c0c035860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c0c0d6e40 0x7f2c0c035860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 -- 172.21.15.8:0/3348153542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c00009420 con 0x7f2c0c0e0100 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2c13fff640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2c14063ef0 tx=0x7f2c140742b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1406bea0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c14094660 con 0x7f2c0c0e0100 2023-10-30T12:45:27.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14094a90 con 0x7f2c0c0e0100 2023-10-30T12:45:27.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe63659c640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe620003980 0x7fe620003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe635d9b640 1 -- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe620005190 msgr2=0x7fe6200141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe620005190 0x7fe6200141a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe635d9b640 1 -- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe620003980 msgr2=0x7fe620003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe620003980 0x7fe620003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56affff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f56b014f910 0x7f56b006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f56b014f910 0x7f56b006b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36312/0 (socket says 172.21.15.8:36312) 2023-10-30T12:45:27.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 -- 172.21.15.8:0/369214172 learned_addr learned my addr 172.21.15.8:0/369214172 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.465+0000 7f56af7fe640 1 --1- 172.21.15.8:0/369214172 >> v1:172.21.15.46:6789/0 conn(0x7f56b00cb790 0x7f56b00c6450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:33032/0 (socket says 172.21.15.8:33032) 2023-10-30T12:45:27.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe635d9b640 1 -- 172.21.15.8:0/2996913430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620003530 con 0x7fe6200047d0 2023-10-30T12:45:27.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe63659c640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe620003980 0x7fe620003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:27.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c17d2d0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56aeffd640 1 --2- 172.21.15.8:0/369214172 >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 0x7f56b006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbee7fc640 1 -- 172.21.15.8:0/435563460 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129034010 0 0) 0x7ffbf00c55f0 con 0x7ffbf0155c40 2023-10-30T12:45:27.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbee7fc640 1 -- 172.21.15.8:0/435563460 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd8003530 con 0x7ffbf0155c40 2023-10-30T12:45:27.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 -- 172.21.15.8:0/369214172 >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 msgr2=0x7f56b006fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c0c17c0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 --2- 172.21.15.8:0/369214172 >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 0x7f56b006fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 -- 172.21.15.8:0/369214172 >> v1:172.21.15.46:6789/0 conn(0x7f56b00cb790 legacy=0x7f56b00c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56affff640 1 -- 172.21.15.8:0/369214172 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00c5970 con 0x7f56b014f910 2023-10-30T12:45:27.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbef7fe640 1 -- 172.21.15.8:0/435563460 >> v2:172.21.15.8:3300/0 conn(0x7ffbf014fc50 msgr2=0x7ffbf0070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbef7fe640 1 --2- 172.21.15.8:0/435563460 >> v2:172.21.15.8:3300/0 conn(0x7ffbf014fc50 0x7ffbf0070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbef7fe640 1 -- 172.21.15.8:0/435563460 >> v1:172.21.15.46:6789/0 conn(0x7ffbf0155c40 legacy=0x7ffbf0159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7ffbef7fe640 1 -- 172.21.15.8:0/435563460 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbf01567b0 con 0x7ffbf00c6320 2023-10-30T12:45:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7fe636d9d640 1 -- 172.21.15.8:0/2996913430 reap_dead start 2023-10-30T12:45:27.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.469+0000 7f56aeffd640 1 --2- 172.21.15.8:0/369214172 >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 0x7f56b006fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:27.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8ed90da640 1 --2- 172.21.15.8:0/1966924003 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009870 tx=0x7f8ed001ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6c69d1fb7d4380c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2c14094bf0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8ecbfff640 1 -- 172.21.15.8:0/1966924003 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00a89b0 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8ecbfff640 1 -- 172.21.15.8:0/1966924003 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed00a8b10 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8ecbfff640 1 -- 172.21.15.8:0/1966924003 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00a8ea0 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c0056d0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2bfffff640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 0x7f2bf8068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c14101ae0 con 0x7f2c0c0e0100 2023-10-30T12:45:27.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2c137fe640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 0x7f2bf8068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7ffbef7fe640 1 --2- 172.21.15.8:0/435563460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf00c6320 0x7ffbf006fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbf006f6e0 tx=0x7ffbe001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff447b3dd4e8a85 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 -- 172.21.15.8:0/1966924003 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 msgr2=0x7f8ecc007060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe620004c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe628009b90 tx=0x7fe62801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561cf458b5698ba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 --2- 172.21.15.8:0/1966924003 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009870 tx=0x7f8ed001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 -- 172.21.15.8:0/1966924003 shutdown_connections 2023-10-30T12:45:27.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f56affff640 1 --2- 172.21.15.8:0/369214172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b014f910 0x7f56b006b500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56b00c25e0 tx=0x7f56a001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97952a97262110f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 --2- 172.21.15.8:0/1966924003 >> v2:172.21.15.46:3300/0 conn(0x7f8ecc00eb70 0x7f8ecc0075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 --2- 172.21.15.8:0/1966924003 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc007060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f2c137fe640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 0x7f2bf8068a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009ac0 tx=0x7f2c08009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 -- 172.21.15.8:0/1966924003 >> 172.21.15.8:0/1966924003 conn(0x7f8ecc00e050 msgr2=0x7f8ecc00e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 -- 172.21.15.8:0/1966924003 shutdown_connections 2023-10-30T12:45:27.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f8eda0dc640 1 -- 172.21.15.8:0/1966924003 wait complete. 2023-10-30T12:45:27.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f56ae7fc640 1 -- 172.21.15.8:0/369214172 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a89f0 con 0x7f56b014f910 2023-10-30T12:45:27.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280a5620 con 0x7fe6200047d0 2023-10-30T12:45:27.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.473+0000 7f56ae7fc640 1 -- 172.21.15.8:0/369214172 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a00a8b50 con 0x7f56b014f910 2023-10-30T12:45:27.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56ae7fc640 1 -- 172.21.15.8:0/369214172 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00b0440 con 0x7f56b014f910 2023-10-30T12:45:27.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 -- 172.21.15.8:0/369214172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b014f910 msgr2=0x7f56b006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 --2- 172.21.15.8:0/369214172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b014f910 0x7f56b006b500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56b00c25e0 tx=0x7f56a001ed30 comp rx=0 tx=0).stop 2023-10-30T12:45:27.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 -- 172.21.15.8:0/369214172 shutdown_connections 2023-10-30T12:45:27.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 --2- 172.21.15.8:0/369214172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b014f910 0x7f56b006b500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 --2- 172.21.15.8:0/369214172 >> v2:172.21.15.46:3300/0 conn(0x7f56b00c7110 0x7f56b006fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 -- 172.21.15.8:0/369214172 >> 172.21.15.8:0/369214172 conn(0x7f56b00cac70 msgr2=0x7f56b00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 Processor -- start 2023-10-30T12:45:27.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 -- 172.21.15.8:0/369214172 shutdown_connections 2023-10-30T12:45:27.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6280a5780 con 0x7fe6200047d0 2023-10-30T12:45:27.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f713759e640 1 Processor -- start 2023-10-30T12:45:27.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63559a640 1 -- 172.21.15.8:0/2996913430 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280a5b10 con 0x7fe6200047d0 2023-10-30T12:45:27.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 Processor -- start 2023-10-30T12:45:27.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f56b5e09640 1 -- 172.21.15.8:0/369214172 wait complete. 2023-10-30T12:45:27.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 -- start start 2023-10-30T12:45:27.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63759e640 1 -- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 msgr2=0x7fe620004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63759e640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe620004c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe628009b90 tx=0x7fe62801ca90 comp rx=0 tx=0).stop 2023-10-30T12:45:27.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63759e640 1 -- 172.21.15.8:0/2996913430 shutdown_connections 2023-10-30T12:45:27.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63759e640 1 --2- 172.21.15.8:0/2996913430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe620004c50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7fe63759e640 1 -- 172.21.15.8:0/2996913430 >> 172.21.15.8:0/2996913430 conn(0x7fe63000e620 msgr2=0x7fe630009cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f713759e640 1 -- start start 2023-10-30T12:45:27.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f463ded6640 1 Processor -- start 2023-10-30T12:45:27.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 -- start start 2023-10-30T12:45:27.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6158153b10 0x7f61580c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61580ca8e0 0x7f615806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 0x7f6158159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6158159900 con 0x7f6158153b10 2023-10-30T12:45:27.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615815d070 con 0x7f61580ca8e0 2023-10-30T12:45:27.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f615ca91640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615815d350 con 0x7f615806c480 2023-10-30T12:45:27.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc00eb70 0x7f8ecc04dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc17c690 con 0x7f8ecc050950 2023-10-30T12:45:27.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0cff70 con 0x7f8ecc00eb70 2023-10-30T12:45:27.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.477+0000 7f8eda0dc640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc17cbc0 con 0x7f8ecc0d8f90 2023-10-30T12:45:27.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61580ca8e0 0x7f615806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56b5e09640 1 Processor -- start 2023-10-30T12:45:27.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f713759e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f7130008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f713759e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f71300e3b60 0x7f71300eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f713759e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713004d930 con 0x7f7130008290 2023-10-30T12:45:27.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f713759e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7130007ff0 con 0x7f71300e3b60 2023-10-30T12:45:27.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f713759e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7130008bb0 con 0x7f713004ce30 2023-10-30T12:45:27.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed98db640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f615659c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6158153b10 0x7f61580c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61580ca8e0 0x7f615806bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50072/0 (socket says 172.21.15.8:50072) 2023-10-30T12:45:27.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 0x7f6158159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 -- 172.21.15.8:0/3806389555 learned_addr learned my addr 172.21.15.8:0/3806389555 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36332/0 (socket says 172.21.15.8:36332) 2023-10-30T12:45:27.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 -- 172.21.15.8:0/3184911769 learned_addr learned my addr 172.21.15.8:0/3184911769 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56b5e09640 1 -- start start 2023-10-30T12:45:27.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 -- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 msgr2=0x7f6158159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 0x7f6158159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 -- 172.21.15.8:0/3806389555 >> v2:172.21.15.8:3300/0 conn(0x7f6158153b10 msgr2=0x7f61580c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 --2- 172.21.15.8:0/3806389555 >> v2:172.21.15.8:3300/0 conn(0x7f6158153b10 0x7f61580c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f614dd9b640 1 -- 172.21.15.8:0/3806389555 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615815d630 con 0x7f61580ca8e0 2023-10-30T12:45:27.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f6156d9d640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 0x7f6158159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:27.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7135d9b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f7130008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f71300e3b60 0x7f71300eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f71300e3b60 0x7f71300eda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36346/0 (socket says 172.21.15.8:36346) 2023-10-30T12:45:27.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 -- 172.21.15.8:0/3598381882 learned_addr learned my addr 172.21.15.8:0/3598381882 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 -- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 msgr2=0x7f8ecc0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc0ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 -- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc00eb70 msgr2=0x7f8ecc04dcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-30T12:45:27.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc00eb70 0x7f8ecc04dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 -- 172.21.15.8:0/3184911769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00094d0 con 0x7f8ecc050950 2023-10-30T12:45:27.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7fe63759e640 1 -- 172.21.15.8:0/2996913430 shutdown_connections 2023-10-30T12:45:27.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ed88d9640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400ebe0 tx=0x7f8ec400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 -- 172.21.15.8:0/3598381882 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 msgr2=0x7f7130008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 --2- 172.21.15.8:0/3598381882 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f7130008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 -- 172.21.15.8:0/3598381882 >> v1:172.21.15.8:6789/0 conn(0x7f7130008290 legacy=0x7f713004df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f7136d9d640 1 -- 172.21.15.8:0/3598381882 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713004dc10 con 0x7f71300e3b60 2023-10-30T12:45:27.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400ecd0 con 0x7f8ecc050950 2023-10-30T12:45:27.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8eda0dc640 1 -- 172.21.15.8:0/3184911769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0d01e0 con 0x7f8ecc050950 2023-10-30T12:45:27.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec400ee30 con 0x7f8ecc050950 2023-10-30T12:45:27.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400ce30 con 0x7f8ecc050950 2023-10-30T12:45:27.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56b5e09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f8eda0dc640 1 -- 172.21.15.8:0/3184911769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0d1550 con 0x7f8ecc050950 2023-10-30T12:45:27.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56b5e09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b00cb790 0x7f56b0148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56af7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56af7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33434/0 (socket says 172.21.15.8:33434) 2023-10-30T12:45:27.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56af7fe640 1 -- 172.21.15.8:0/3333416700 learned_addr learned my addr 172.21.15.8:0/3333416700 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 0x7f56b01657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 0x7f56b01657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 -- start start 2023-10-30T12:45:27.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 0x7f463806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f463806c480 0x7f4638159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463815d070 con 0x7f4638153b10 2023-10-30T12:45:27.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4638159900 con 0x7f463806c480 2023-10-30T12:45:27.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.481+0000 7f463ded6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380ca000 con 0x7f46380c9b90 2023-10-30T12:45:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01535d0 con 0x7f56b00cb790 2023-10-30T12:45:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0146110 con 0x7f56b014f910 2023-10-30T12:45:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0153800 con 0x7f56b00c7110 2023-10-30T12:45:27.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56aeffd640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b00cb790 0x7f56b0148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc049d70 con 0x7f8ecc050950 2023-10-30T12:45:27.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f463806c480 0x7f4638159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 -- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 msgr2=0x7f56b01465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 -- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b00cb790 msgr2=0x7f56b0148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b00cb790 0x7f56b0148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00094d0 con 0x7f56b014f910 2023-10-30T12:45:27.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56af7fe640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:27.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f614dd9b640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f615806bf40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61580b1dd0 tx=0x7f614001c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88597d6db50fa18e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f463806c480 0x7f4638159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36360/0 (socket says 172.21.15.8:36360) 2023-10-30T12:45:27.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 -- 172.21.15.8:0/1438081053 learned_addr learned my addr 172.21.15.8:0/1438081053 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4636ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 0x7f463806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f56affff640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 0x7f56b01657b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b00740b0 tx=0x7f56a001bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f7136d9d640 1 --2- 172.21.15.8:0/3598381882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300eda20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7138074c50 tx=0x7f7138099cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36d21bb15d125e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 -- 172.21.15.8:0/1438081053 >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 msgr2=0x7f463806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 --2- 172.21.15.8:0/1438081053 >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 0x7f463806bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 -- 172.21.15.8:0/1438081053 >> v1:172.21.15.8:6789/0 conn(0x7f4638153b10 legacy=0x7f46380ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:27.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7fe63759e640 1 -- 172.21.15.8:0/2996913430 wait complete. 2023-10-30T12:45:27.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ec4013470 con 0x7f8ecc050950 2023-10-30T12:45:27.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4637fff640 1 -- 172.21.15.8:0/1438081053 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463815d350 con 0x7f463806c480 2023-10-30T12:45:27.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7fe63759e640 1 Processor -- start 2023-10-30T12:45:27.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f4636ffd640 1 --2- 172.21.15.8:0/1438081053 >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 0x7f463806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:27.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7fe63759e640 1 -- start start 2023-10-30T12:45:27.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.485+0000 7f8ec9ffb640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 0x7f8eb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbee7fc640 1 -- 172.21.15.8:0/435563460 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00ab670 con 0x7ffbf00c6320 2023-10-30T12:45:27.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbee7fc640 1 -- 172.21.15.8:0/435563460 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe00ab7d0 con 0x7ffbf00c6320 2023-10-30T12:45:27.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbee7fc640 1 -- 172.21.15.8:0/435563460 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00abb60 con 0x7ffbf00c6320 2023-10-30T12:45:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ec4083fa0 con 0x7f8ecc050950 2023-10-30T12:45:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f6155d9b640 1 -- 172.21.15.8:0/3806389555 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400ac630 con 0x7f61580ca8e0 2023-10-30T12:45:27.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f6155d9b640 1 -- 172.21.15.8:0/3806389555 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61400ac790 con 0x7f61580ca8e0 2023-10-30T12:45:27.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f6155d9b640 1 -- 172.21.15.8:0/3806389555 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400acb20 con 0x7f61580ca8e0 2023-10-30T12:45:27.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2c140cf000 con 0x7f2c0c0e0100 2023-10-30T12:45:27.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00b1e90 con 0x7f56b014f910 2023-10-30T12:45:27.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a00b9440 con 0x7f56b014f910 2023-10-30T12:45:27.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f8ed90da640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 0x7f8eb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00b97f0 con 0x7f56b014f910 2023-10-30T12:45:27.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01457e0 con 0x7f56b014f910 2023-10-30T12:45:27.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf00c6320 msgr2=0x7ffbf006fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 --2- 172.21.15.8:0/435563460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf00c6320 0x7ffbf006fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbf006f6e0 tx=0x7ffbe001c470 comp rx=0 tx=0).stop 2023-10-30T12:45:27.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b0145da0 con 0x7f56b014f910 2023-10-30T12:45:27.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 -- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 msgr2=0x7f615806bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f615806bf40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61580b1dd0 tx=0x7f614001c500 comp rx=0 tx=0).stop 2023-10-30T12:45:27.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 -- 172.21.15.8:0/3806389555 shutdown_connections 2023-10-30T12:45:27.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f615806c480 0x7f6158159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 --2- 172.21.15.8:0/3806389555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f615806bf40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 --2- 172.21.15.8:0/3806389555 >> v2:172.21.15.8:3300/0 conn(0x7f6158153b10 0x7f61580c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 -- 172.21.15.8:0/3806389555 >> 172.21.15.8:0/3806389555 conn(0x7f615806a240 msgr2=0x7f615806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 -- 172.21.15.8:0/3806389555 shutdown_connections 2023-10-30T12:45:27.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f615ca91640 1 -- 172.21.15.8:0/3806389555 wait complete. 2023-10-30T12:45:27.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 shutdown_connections 2023-10-30T12:45:27.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 --2- 172.21.15.8:0/435563460 >> v2:172.21.15.8:3300/0 conn(0x7ffbf014fc50 0x7ffbf0070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 --2- 172.21.15.8:0/435563460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf00c6320 0x7ffbf006fdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 >> 172.21.15.8:0/435563460 conn(0x7ffbf00cac70 msgr2=0x7ffbf00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b0066510 con 0x7f56b014f910 2023-10-30T12:45:27.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.489+0000 7f8ed90da640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 0x7f8eb8068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00b2000 tx=0x7f8ed00aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 shutdown_connections 2023-10-30T12:45:27.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7ffbf613e640 1 -- 172.21.15.8:0/435563460 wait complete. 2023-10-30T12:45:27.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe630175db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe630182b00 0x7fe630182f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630050800 con 0x7fe6200047d0 2023-10-30T12:45:27.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630182670 con 0x7fe6301782b0 2023-10-30T12:45:27.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63759e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6301834c0 con 0x7fe630182b00 2023-10-30T12:45:27.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 Processor -- start 2023-10-30T12:45:27.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50108/0 (socket says 172.21.15.8:50108) 2023-10-30T12:45:27.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 -- 172.21.15.8:0/2878170511 learned_addr learned my addr 172.21.15.8:0/2878170511 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 -- start start 2023-10-30T12:45:27.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe63659c640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe630175db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe636d9d640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe630182b00 0x7fe630182f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 -- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe630182b00 msgr2=0x7fe630182f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe630182b00 0x7fe630182f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 -- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 msgr2=0x7fe630175db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe630175db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 -- 172.21.15.8:0/2878170511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6301768d0 con 0x7fe6301782b0 2023-10-30T12:45:27.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7fe635d9b640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe628009b90 tx=0x7fe628019830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7ffbf613e640 1 Processor -- start 2023-10-30T12:45:27.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7ffbf613e640 1 -- start start 2023-10-30T12:45:27.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56a00b7070 con 0x7f56b014f910 2023-10-30T12:45:27.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f5697fff640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 0x7f569c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56a0121840 con 0x7f56b014f910 2023-10-30T12:45:27.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f615806c480 0x7f61581490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f61580ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158157dd0 con 0x7f615806c480 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158153390 con 0x7f61580ca8e0 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615ca91640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61581f5f70 con 0x7f6158153b10 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f615659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f615806c480 0x7f61581490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33444/0 (socket says 172.21.15.8:33444) 2023-10-30T12:45:27.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f6156d9d640 1 -- 172.21.15.8:0/3804276556 learned_addr learned my addr 172.21.15.8:0/3804276556 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.493+0000 7f614dd9b640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f61580ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f4637fff640 1 --2- 172.21.15.8:0/1438081053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638159d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46380b1dd0 tx=0x7f463001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da4e848b48942a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 -- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 msgr2=0x7f61580ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f61580ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 -- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f615806c480 msgr2=0x7f61581490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f615806c480 0x7f61581490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 -- 172.21.15.8:0/3804276556 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150009420 con 0x7f6158153b10 2023-10-30T12:45:27.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ffbf00c6320 0x7ffbf00ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 0x7ffbf0148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0162bb0 con 0x7ffbf0155c40 2023-10-30T12:45:27.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0157990 con 0x7ffbf014fc50 2023-10-30T12:45:27.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbf613e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0157c70 con 0x7ffbf00c6320 2023-10-30T12:45:27.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f6156d9d640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f615000e8c0 tx=0x7f615000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbef7fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ffbf00c6320 0x7ffbf00ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150004020 con 0x7f6158153b10 2023-10-30T12:45:27.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61500041a0 con 0x7f6158153b10 2023-10-30T12:45:27.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000c940 con 0x7f6158153b10 2023-10-30T12:45:27.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400094d0 con 0x7f6158153b10 2023-10-30T12:45:27.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50124/0 (socket says 172.21.15.8:50124) 2023-10-30T12:45:27.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 -- 172.21.15.8:0/1810639131 learned_addr learned my addr 172.21.15.8:0/1810639131 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f56af7fe640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 0x7f569c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6158151c50 con 0x7f6158153b10 2023-10-30T12:45:27.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f46367fc640 1 -- 172.21.15.8:0/1438081053 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300a5710 con 0x7f463806c480 2023-10-30T12:45:27.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeffff640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 0x7ffbf0148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f713559a640 1 -- 172.21.15.8:0/3598381882 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138072040 con 0x7f71300e3b60 2023-10-30T12:45:27.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6158067540 con 0x7f6158153b10 2023-10-30T12:45:27.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 -- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ffbf00c6320 msgr2=0x7ffbf00ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ffbf00c6320 0x7ffbf00ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 -- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 msgr2=0x7ffbf0148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f713559a640 1 -- 172.21.15.8:0/3598381882 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138123a20 con 0x7f71300e3b60 2023-10-30T12:45:27.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f713559a640 1 -- 172.21.15.8:0/3598381882 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138123e50 con 0x7f71300e3b60 2023-10-30T12:45:27.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f46367fc640 1 -- 172.21.15.8:0/1438081053 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300a5870 con 0x7f463806c480 2023-10-30T12:45:27.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f46367fc640 1 -- 172.21.15.8:0/1438081053 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300a5c00 con 0x7f463806c480 2023-10-30T12:45:27.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 0x7ffbf0148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8ecc049d70 con 0x7f8ecc050950 2023-10-30T12:45:27.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7ffbeeffd640 1 -- 172.21.15.8:0/1810639131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe00094d0 con 0x7ffbf014fc50 2023-10-30T12:45:27.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.497+0000 7f56af7fe640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 0x7f569c0689b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56a8007fd0 tx=0x7f56a80017c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7ffbeffff640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 0x7ffbf0148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:27.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 -- 172.21.15.8:0/3598381882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 msgr2=0x7f71300eda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 --2- 172.21.15.8:0/3598381882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300eda20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7138074c50 tx=0x7f7138099cb0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f615000caa0 con 0x7f6158153b10 2023-10-30T12:45:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 -- 172.21.15.8:0/3598381882 shutdown_connections 2023-10-30T12:45:27.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 --2- 172.21.15.8:0/3598381882 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300eda20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 --2- 172.21.15.8:0/3598381882 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f7130008670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 -- 172.21.15.8:0/3598381882 >> 172.21.15.8:0/3598381882 conn(0x7f713000a240 msgr2=0x7f713000e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7ffbeeffd640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e930 tx=0x7ffbe400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 -- 172.21.15.8:0/3598381882 shutdown_connections 2023-10-30T12:45:27.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f713759e640 1 -- 172.21.15.8:0/3598381882 wait complete. 2023-10-30T12:45:27.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f614f7fe640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 0x7f6138068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6150084480 con 0x7f6158153b10 2023-10-30T12:45:27.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f615659c640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 0x7f6138068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f615659c640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 0x7f6138068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6148007380 tx=0x7f614800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 -- 172.21.15.8:0/1438081053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 msgr2=0x7f4638159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 --2- 172.21.15.8:0/1438081053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638159d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46380b1dd0 tx=0x7f463001c940 comp rx=0 tx=0).stop 2023-10-30T12:45:27.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 -- 172.21.15.8:0/1438081053 shutdown_connections 2023-10-30T12:45:27.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 --2- 172.21.15.8:0/1438081053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638159d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 --2- 172.21.15.8:0/1438081053 >> v2:172.21.15.46:3300/0 conn(0x7f46380c9b90 0x7f463806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 -- 172.21.15.8:0/1438081053 >> 172.21.15.8:0/1438081053 conn(0x7f463806a240 msgr2=0x7f463806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 -- 172.21.15.8:0/1438081053 shutdown_connections 2023-10-30T12:45:27.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.501+0000 7f463ded6640 1 -- 172.21.15.8:0/1438081053 wait complete. 2023-10-30T12:45:27.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 Processor -- start 2023-10-30T12:45:27.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 -- start start 2023-10-30T12:45:27.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 Processor -- start 2023-10-30T12:45:27.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 -- start start 2023-10-30T12:45:27.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f713004b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300d9b10 con 0x7f71300e3b60 2023-10-30T12:45:27.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300d9200 con 0x7f7130008290 2023-10-30T12:45:27.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713759e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300daaa0 con 0x7f713004ce30 2023-10-30T12:45:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 0x7f463813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4638153b10 0x7f4638145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46381f6290 con 0x7f463806c480 2023-10-30T12:45:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638144570 con 0x7f46380c9b90 2023-10-30T12:45:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f463ded6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4638144290 con 0x7f4638153b10 2023-10-30T12:45:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50126/0 (socket says 172.21.15.8:50126) 2023-10-30T12:45:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 -- 172.21.15.8:0/38056613 learned_addr learned my addr 172.21.15.8:0/38056613 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f7135d9b640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f713004b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f4636ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 0x7f463813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f4636ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 0x7f463813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50138/0 (socket says 172.21.15.8:50138) 2023-10-30T12:45:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f4636ffd640 1 -- 172.21.15.8:0/3948330010 learned_addr learned my addr 172.21.15.8:0/3948330010 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36396/0 (socket says 172.21.15.8:36396) 2023-10-30T12:45:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f4637fff640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4638153b10 0x7f4638145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 -- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 msgr2=0x7f713004b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f713004b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 -- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 msgr2=0x7f71300d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f7136d9d640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300d6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280b6870 con 0x7fe6301782b0 2023-10-30T12:45:27.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f713659c640 1 -- 172.21.15.8:0/38056613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300dad60 con 0x7f7130008290 2023-10-30T12:45:27.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 -- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4638153b10 msgr2=0x7f4638145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4638153b10 0x7f4638145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 -- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 msgr2=0x7f463813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.505+0000 7f46377fe640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 0x7f463813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6280b69d0 con 0x7fe6301782b0 2023-10-30T12:45:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe630184c90 con 0x7fe6301782b0 2023-10-30T12:45:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe630182300 con 0x7fe6301782b0 2023-10-30T12:45:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280b6d60 con 0x7fe6301782b0 2023-10-30T12:45:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4009690 con 0x7ffbf014fc50 2023-10-30T12:45:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe400ca00 con 0x7ffbf014fc50 2023-10-30T12:45:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400cd70 con 0x7ffbf014fc50 2023-10-30T12:45:27.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56a00eed50 con 0x7f56b014f910 2023-10-30T12:45:27.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f46377fe640 1 -- 172.21.15.8:0/3948330010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4630009b30 con 0x7f463806c480 2023-10-30T12:45:27.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f46377fe640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f70 tx=0x7f462c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0052d0 con 0x7f463806c480 2023-10-30T12:45:27.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c005430 con 0x7f463806c480 2023-10-30T12:45:27.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c003840 con 0x7f463806c480 2023-10-30T12:45:27.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf0147c30 con 0x7ffbf014fc50 2023-10-30T12:45:27.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46381f5960 con 0x7f463806c480 2023-10-30T12:45:27.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf01481f0 con 0x7ffbf014fc50 2023-10-30T12:45:27.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46381f5f20 con 0x7f463806c480 2023-10-30T12:45:27.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe630182820 con 0x7fe6301782b0 2023-10-30T12:45:27.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4638067540 con 0x7f463806c480 2023-10-30T12:45:27.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe6280b4070 con 0x7fe6301782b0 2023-10-30T12:45:27.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf004e170 con 0x7ffbf014fc50 2023-10-30T12:45:27.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f713659c640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f712800c4c0 tx=0x7f7128001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128005480 con 0x7f7130008290 2023-10-30T12:45:27.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128003600 con 0x7f7130008290 2023-10-30T12:45:27.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.509+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128003990 con 0x7f7130008290 2023-10-30T12:45:27.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300db040 con 0x7f7130008290 2023-10-30T12:45:27.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300d9730 con 0x7f7130008290 2023-10-30T12:45:27.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7fe626ffd640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 0x7fe618068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe62811def0 con 0x7fe6301782b0 2023-10-30T12:45:27.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7fe63659c640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 0x7fe618068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7130009980 con 0x7f7130008290 2023-10-30T12:45:27.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7fe63659c640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 0x7fe618068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe638000b70 tx=0x7fe638075890 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f462c0039a0 con 0x7f463806c480 2023-10-30T12:45:27.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f461ffff640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 0x7f4624068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f462c083410 con 0x7f463806c480 2023-10-30T12:45:27.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7f4636ffd640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 0x7f4624068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffbe4013470 con 0x7ffbf014fc50 2023-10-30T12:45:27.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.513+0000 7ffbd7fff640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 0x7ffbdc0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.517+0000 7f4636ffd640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 0x7f4624068a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f463815a0a0 tx=0x7f4628008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.517+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6150051a00 con 0x7f6158153b10 2023-10-30T12:45:27.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.517+0000 7ffbef7fe640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 0x7ffbdc0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.517+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbe4055250 con 0x7ffbf014fc50 2023-10-30T12:45:27.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.517+0000 7ffbef7fe640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 0x7ffbdc0689e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00af000 tx=0x7ffbe00b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.525+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe630182820 con 0x7fe6301782b0 2023-10-30T12:45:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.525+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7128003af0 con 0x7f7130008290 2023-10-30T12:45:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.529+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f462c050920 con 0x7f463806c480 2023-10-30T12:45:27.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.529+0000 7f7126ffd640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:27.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.529+0000 7f7135d9b640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:27.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.529+0000 7f7135d9b640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 0x7f7118068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71300e6ab0 tx=0x7f712c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:27.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.537+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffbe4051d20 con 0x7ffbf014fc50 2023-10-30T12:45:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.537+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7128054290 con 0x7f7130008290 2023-10-30T12:45:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.549+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71280505d0 con 0x7f7130008290 2023-10-30T12:45:27.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.821+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2c0c17cf80 con 0x7f2c0c0e0100 2023-10-30T12:45:27.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.821+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f61580ca170 con 0x7f6158153b10 2023-10-30T12:45:27.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.821+0000 7f2bfffff640 1 -- 172.21.15.8:0/3348153542 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c14099020 con 0x7f2c0c0e0100 2023-10-30T12:45:27.824 INFO:teuthology.orchestra.run.smithi008.stdout:55834574853 2023-10-30T12:45:27.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f614f7fe640 1 -- 172.21.15.8:0/3804276556 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6150020090 con 0x7f6158153b10 2023-10-30T12:45:27.826 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 msgr2=0x7f2bf8068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 0x7f2bf8068a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009ac0 tx=0x7f2c08009190 comp rx=0 tx=0).stop 2023-10-30T12:45:27.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 msgr2=0x7f2c0c036630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2c14063ef0 tx=0x7f2c140742b0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 msgr2=0x7f6138068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 0x7f6138068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6148007380 tx=0x7f614800b040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 msgr2=0x7f6158142ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f615000e8c0 tx=0x7f615000ed80 comp rx=0 tx=0).stop 2023-10-30T12:45:27.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 shutdown_connections 2023-10-30T12:45:27.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2bf80665a0 0x7f2bf8068a60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0e0100 0x7f2c0c036630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c0c0d6e40 0x7f2c0c035860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 --2- 172.21.15.8:0/3348153542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c0c00ebe0 0x7f2c0c0d5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 >> 172.21.15.8:0/3348153542 conn(0x7f2c0c00a240 msgr2=0x7f2c0c047d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 shutdown_connections 2023-10-30T12:45:27.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f61380664a0 0x7f6138068960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61580ca8e0 0x7f61580ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6158153b10 0x7f6158142ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 --2- 172.21.15.8:0/3804276556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f615806c480 0x7f61581490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.825+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 >> 172.21.15.8:0/3804276556 conn(0x7f615806a240 msgr2=0x7f6158158260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.829+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 shutdown_connections 2023-10-30T12:45:27.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.829+0000 7f615ca91640 1 -- 172.21.15.8:0/3804276556 wait complete. 2023-10-30T12:45:27.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.829+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 shutdown_connections 2023-10-30T12:45:27.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.829+0000 7f2c19d80640 1 -- 172.21.15.8:0/3348153542 wait complete. 2023-10-30T12:45:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.841+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8ecc049d70 con 0x7f8ecc050950 2023-10-30T12:45:27.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.841+0000 7f8ec9ffb640 1 -- 172.21.15.8:0/3184911769 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ec4017020 con 0x7f8ecc050950 2023-10-30T12:45:27.844 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:27.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 msgr2=0x7f8eb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 0x7f8eb8068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00b2000 tx=0x7f8ed00aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:27.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 msgr2=0x7f8ecc0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400ebe0 tx=0x7f8ec400c520 comp rx=0 tx=0).stop 2023-10-30T12:45:27.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 shutdown_connections 2023-10-30T12:45:27.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8eb80664a0 0x7f8eb8068960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ecc0d8f90 0x7f8ecc0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ecc050950 0x7f8ecc0ce660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 --2- 172.21.15.8:0/3184911769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc00eb70 0x7f8ecc04dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 >> 172.21.15.8:0/3184911769 conn(0x7f8ecc00e050 msgr2=0x7f8ecc0d7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 shutdown_connections 2023-10-30T12:45:27.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.845+0000 7f8ebf7fe640 1 -- 172.21.15.8:0/3184911769 wait complete. 2023-10-30T12:45:27.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.849+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f56b004e170 con 0x7f56b014f910 2023-10-30T12:45:27.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.849+0000 7f5697fff640 1 -- 172.21.15.8:0/3333416700 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56a011fe60 con 0x7f56b014f910 2023-10-30T12:45:27.852 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.849+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe630182820 con 0x7fe6301782b0 2023-10-30T12:45:27.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7fe626ffd640 1 -- 172.21.15.8:0/2878170511 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe630182820 con 0x7fe6301782b0 2023-10-30T12:45:27.856 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 msgr2=0x7f569c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 0x7f569c0689b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56a8007fd0 tx=0x7f56a80017c0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 msgr2=0x7f56b01657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 0x7f56b01657b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b00740b0 tx=0x7f56a001bad0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 shutdown_connections 2023-10-30T12:45:27.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f569c0664f0 0x7f569c0689b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b014f910 0x7f56b01657b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56b00cb790 0x7f56b0148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 --2- 172.21.15.8:0/3333416700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56b00c7110 0x7f56b01465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 >> 172.21.15.8:0/3333416700 conn(0x7f56b00cac70 msgr2=0x7f56b014eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 shutdown_connections 2023-10-30T12:45:27.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.853+0000 7f56b5e09640 1 -- 172.21.15.8:0/3333416700 wait complete. 2023-10-30T12:45:27.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 msgr2=0x7fe618068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 0x7fe618068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe638000b70 tx=0x7fe638075890 comp rx=0 tx=0).stop 2023-10-30T12:45:27.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 msgr2=0x7fe6300518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe628009b90 tx=0x7fe628019830 comp rx=0 tx=0).stop 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 shutdown_connections 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6180664d0 0x7fe618068990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6301782b0 0x7fe6300518f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe630182b00 0x7fe630182f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 --2- 172.21.15.8:0/2878170511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200047d0 0x7fe630175db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 >> 172.21.15.8:0/2878170511 conn(0x7fe63000e620 msgr2=0x7fe6300d6020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 shutdown_connections 2023-10-30T12:45:27.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7fe63759e640 1 -- 172.21.15.8:0/2878170511 wait complete. 2023-10-30T12:45:27.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f46380cef30 con 0x7f463806c480 2023-10-30T12:45:27.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.857+0000 7f461ffff640 1 -- 172.21.15.8:0/3948330010 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f462c053f40 con 0x7f463806c480 2023-10-30T12:45:27.863 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 msgr2=0x7f4624068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 0x7f4624068a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f463815a0a0 tx=0x7f4628008040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 msgr2=0x7f4638157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f70 tx=0x7f462c00c820 comp rx=0 tx=0).stop 2023-10-30T12:45:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 shutdown_connections 2023-10-30T12:45:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4624066570 0x7f4624068a30 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f463815a0a0 tx=0x7f4628008040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380c9b90 0x7f463813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4638153b10 0x7f4638145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 --2- 172.21.15.8:0/3948330010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f463806c480 0x7f4638157990 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f70 tx=0x7f462c00c820 comp rx=0 tx=0).stop 2023-10-30T12:45:27.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.861+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 >> 172.21.15.8:0/3948330010 conn(0x7f463806a240 msgr2=0x7f46380cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-30T12:45:27.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-30T12:45:27.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.865+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 shutdown_connections 2023-10-30T12:45:27.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.865+0000 7f463ded6640 1 -- 172.21.15.8:0/3948330010 wait complete. 2023-10-30T12:45:27.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.877+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffbf0073d50 con 0x7ffbf014fc50 2023-10-30T12:45:27.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.877+0000 7ffbd7fff640 1 -- 172.21.15.8:0/1810639131 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbe4017090 con 0x7ffbf014fc50 2023-10-30T12:45:27.880 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:27.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-10-30T12:45:27.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 msgr2=0x7ffbdc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 0x7ffbdc0689e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00af000 tx=0x7ffbe00b6040 comp rx=0 tx=0).stop 2023-10-30T12:45:27.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 msgr2=0x7ffbf01489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e930 tx=0x7ffbe400edf0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 shutdown_connections 2023-10-30T12:45:27.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ffbdc066520 0x7ffbdc0689e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbf014fc50 0x7ffbf01489f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbf0155c40 0x7ffbf0148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 --2- 172.21.15.8:0/1810639131 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ffbf00c6320 0x7ffbf00ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 >> 172.21.15.8:0/1810639131 conn(0x7ffbf00cac70 msgr2=0x7ffbf0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 shutdown_connections 2023-10-30T12:45:27.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7ffbf613e640 1 -- 172.21.15.8:0/1810639131 wait complete. 2023-10-30T12:45:27.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.881+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71300056d0 con 0x7f7130008290 2023-10-30T12:45:27.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f7126ffd640 1 -- 172.21.15.8:0/38056613 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7128017020 con 0x7f7130008290 2023-10-30T12:45:27.886 INFO:teuthology.orchestra.run.smithi008.stdout:55834574853 2023-10-30T12:45:27.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 msgr2=0x7f7118068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 0x7f7118068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71300e6ab0 tx=0x7f712c009190 comp rx=0 tx=0).stop 2023-10-30T12:45:27.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 msgr2=0x7f71300db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:27.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f712800c4c0 tx=0x7f7128001e80 comp rx=0 tx=0).stop 2023-10-30T12:45:27.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 shutdown_connections 2023-10-30T12:45:27.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71300e3b60 0x7f71300d6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f713004ce30 0x7f713004b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 --2- 172.21.15.8:0/38056613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7130008290 0x7f71300db5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:27.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 >> 172.21.15.8:0/38056613 conn(0x7f713000a240 msgr2=0x7f713000de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:27.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 shutdown_connections 2023-10-30T12:45:27.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:27.885+0000 7f713759e640 1 -- 172.21.15.8:0/38056613 wait complete. 2023-10-30T12:45:27.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-30T12:45:27.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-10-30T12:45:27.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-10-30T12:45:27.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-30T12:45:27.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-30T12:45:28.870 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T12:45:28.870 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T12:45:28.881 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T12:45:28.897 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T12:45:28.897 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T12:45:28.900 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T12:45:28.909 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T12:45:28.913 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T12:45:29.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.093+0000 7f86d0009640 1 Processor -- start 2023-10-30T12:45:29.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.093+0000 7f86d0009640 1 -- start start 2023-10-30T12:45:29.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f86c80c9ef0 0x7f86c80cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86c80cf4e0 0x7f86c815f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c80cfb10 con 0x7f86c80c9ef0 2023-10-30T12:45:29.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c80cbd20 con 0x7f86c80cf4e0 2023-10-30T12:45:29.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86d0009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80cbfc0 con 0x7f86c815e810 2023-10-30T12:45:29.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:33472/0 (socket says 172.21.15.8:33472) 2023-10-30T12:45:29.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 -- 172.21.15.8:0/2944871180 learned_addr learned my addr 172.21.15.8:0/2944871180 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86ce5a6640 1 --2- 172.21.15.8:0/2944871180 >> v2:172.21.15.46:3300/0 conn(0x7f86c80cf4e0 0x7f86c815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 -- 172.21.15.8:0/2944871180 >> v2:172.21.15.46:3300/0 conn(0x7f86c80cf4e0 msgr2=0x7f86c815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 --2- 172.21.15.8:0/2944871180 >> v2:172.21.15.46:3300/0 conn(0x7f86c80cf4e0 0x7f86c815f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 -- 172.21.15.8:0/2944871180 >> v2:172.21.15.8:3300/0 conn(0x7f86c80c9ef0 msgr2=0x7f86c80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 --2- 172.21.15.8:0/2944871180 >> v2:172.21.15.8:3300/0 conn(0x7f86c80c9ef0 0x7f86c80cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.097+0000 7f86cdda5640 1 -- 172.21.15.8:0/2944871180 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80cc2a0 con 0x7f86c815e810 2023-10-30T12:45:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86cdda5640 1 --2- 172.21.15.8:0/2944871180 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b5080 tx=0x7f86c401ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac5ceff561be7a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86ccda3640 1 -- 172.21.15.8:0/2944871180 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40a89b0 con 0x7f86c815e810 2023-10-30T12:45:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86ccda3640 1 -- 172.21.15.8:0/2944871180 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c40a8b10 con 0x7f86c815e810 2023-10-30T12:45:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86ccda3640 1 -- 172.21.15.8:0/2944871180 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40a8ea0 con 0x7f86c815e810 2023-10-30T12:45:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 -- 172.21.15.8:0/2944871180 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 msgr2=0x7f86c80ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 --2- 172.21.15.8:0/2944871180 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b5080 tx=0x7f86c401ed10 comp rx=0 tx=0).stop 2023-10-30T12:45:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 -- 172.21.15.8:0/2944871180 shutdown_connections 2023-10-30T12:45:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 --2- 172.21.15.8:0/2944871180 >> v2:172.21.15.46:3300/0 conn(0x7f86c80cf4e0 0x7f86c815f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 --2- 172.21.15.8:0/2944871180 >> v2:172.21.15.8:3300/0 conn(0x7f86c80c9ef0 0x7f86c80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 --2- 172.21.15.8:0/2944871180 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c815e810 0x7f86c80ca420 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b5080 tx=0x7f86c401ed10 comp rx=0 tx=0).stop 2023-10-30T12:45:29.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.105+0000 7f86d0009640 1 -- 172.21.15.8:0/2944871180 >> 172.21.15.8:0/2944871180 conn(0x7f86c806bf60 msgr2=0x7f86c806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f2000c46640 1 Processor -- start 2023-10-30T12:45:29.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f2000c46640 1 -- start start 2023-10-30T12:45:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f86d0009640 1 -- 172.21.15.8:0/2944871180 shutdown_connections 2023-10-30T12:45:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f86d0009640 1 -- 172.21.15.8:0/2944871180 wait complete. 2023-10-30T12:45:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f86d0009640 1 Processor -- start 2023-10-30T12:45:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f2000c46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1ffc06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f86d0009640 1 -- start start 2023-10-30T12:45:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.109+0000 7f2000c46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1ffc15d880 0x7f1ffc1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f2000c46640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc153c80 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f2000c46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc15d460 con 0x7f1ffc15d880 2023-10-30T12:45:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f2000c46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15dcc0 con 0x7f1ffc1538a0 2023-10-30T12:45:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ffad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ffc0c6cb0 0x7f1ffc0c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45084/0 (socket says 172.21.15.8:45084) 2023-10-30T12:45:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ffad9d640 1 -- 172.21.15.8:0/3905507969 learned_addr learned my addr 172.21.15.8:0/3905507969 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1ffc06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/3905507969 >> v2:172.21.15.8:3300/0 conn(0x7f1ffc15d880 0x7f1ffc1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021339188 0 0) 0x7f1ffc153c80 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe4003530 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c80c9ef0 0x7f86c81f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c80c8c40 0x7f86c80c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55776/0 (socket says 172.21.15.8:55776) 2023-10-30T12:45:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 -- 172.21.15.8:0/2740237304 learned_addr learned my addr 172.21.15.8:0/2740237304 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cdda5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c80c9ef0 0x7f86c81f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c81f3d10 con 0x7f86c80cf4e0 2023-10-30T12:45:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c3860 con 0x7f86c80c8c40 2023-10-30T12:45:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86ce5a6640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c80c8c40 0x7f86c80c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c9480 con 0x7f86c80c9ef0 2023-10-30T12:45:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 <== mon.1 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605076381 0 0) 0x7f1fe4003530 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 msgr2=0x7f1ffc06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1ffc06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 >> v2:172.21.15.8:3300/0 conn(0x7f1ffc15d880 msgr2=0x7f1ffc1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 --2- 172.21.15.8:0/3905507969 >> v2:172.21.15.8:3300/0 conn(0x7f1ffc15d880 0x7f1ffc1597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc15a060 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 <== mon.1 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f1ff4001a00 con 0x7f1ffc0c6cb0 2023-10-30T12:45:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 >> v1:172.21.15.46:6789/0 conn(0x7f1ffc0c6cb0 legacy=0x7f1ffc0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe4003980 0x7f1fe4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fe40047d0 0x7f1fe4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1fe4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15d460 con 0x7f1fe40047d0 2023-10-30T12:45:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15dcc0 con 0x7f1fe4003980 2023-10-30T12:45:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4014250 con 0x7f1ffc1538a0 2023-10-30T12:45:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/3905507969 >> v2:172.21.15.8:3300/0 conn(0x7f1ffc15d880 0x7f1ffc1597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:29.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe4003980 0x7f1fe4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 -- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c80c9ef0 msgr2=0x7f86c81f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c80c9ef0 0x7f86c81f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 -- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c80c8c40 msgr2=0x7f86c80c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c80c8c40 0x7f86c80c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.113+0000 7f86cd5a4640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40094d0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86cd5a4640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86b800b550 tx=0x7f86b800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffad9d640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1fe4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fe40047d0 0x7f1fe4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffa59c640 1 -- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 msgr2=0x7f1fe4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010870 con 0x7f86c80cf4e0 2023-10-30T12:45:29.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80109f0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010da0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc1538a0 0x7f1fe4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffa59c640 1 -- 172.21.15.8:0/3905507969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fe40047d0 msgr2=0x7f1fe4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fe40047d0 0x7f1fe4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ffa59c640 1 -- 172.21.15.8:0/3905507969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4003530 con 0x7f1fe4003980 2023-10-30T12:45:29.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fe40047d0 0x7f1fe4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:29.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c80c91a0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.117+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c80c3210 con 0x7f86c80cf4e0 2023-10-30T12:45:29.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe4003980 0x7f1fe4003da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00ba30 tx=0x7f1fec01c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f723daf8c984d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.121+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86b800f3c0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.121+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c8067cb0 con 0x7f86c80cf4e0 2023-10-30T12:45:29.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0ad630 con 0x7f1fe4003980 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f1ff959a640 1 -- 172.21.15.8:0/3905507969 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec0ad790 con 0x7f1fe4003980 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f2000c46640 1 -- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe4003980 msgr2=0x7f1fe4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f2000c46640 1 --2- 172.21.15.8:0/3905507969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe4003980 0x7f1fe4003da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00ba30 tx=0x7f1fec01c710 comp rx=0 tx=0).stop 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f1ffad9d640 1 -- 172.21.15.8:0/3905507969 reap_dead start 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f2000c46640 1 -- 172.21.15.8:0/3905507969 shutdown_connections 2023-10-30T12:45:29.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f2000c46640 1 -- 172.21.15.8:0/3905507969 >> 172.21.15.8:0/3905507969 conn(0x7f1ffc06d810 msgr2=0x7f1ffc06dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f86be7fc640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 0x7f86ac0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.125+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86b8083d00 con 0x7f86c80cf4e0 2023-10-30T12:45:29.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.129+0000 7f86cdda5640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 0x7f86ac0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.129+0000 7f2000c46640 1 -- 172.21.15.8:0/3905507969 shutdown_connections 2023-10-30T12:45:29.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.133+0000 7f86cdda5640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 0x7f86ac0689e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b2000 tx=0x7f86c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.133+0000 7f2000c46640 1 -- 172.21.15.8:0/3905507969 wait complete. 2023-10-30T12:45:29.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.133+0000 7f2000c46640 1 Processor -- start 2023-10-30T12:45:29.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.133+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86b8087030 con 0x7f86c80cf4e0 2023-10-30T12:45:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.133+0000 7f2000c46640 1 -- start start 2023-10-30T12:45:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 0x7f1ffc1eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 0x7f1ffc14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ff9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ff9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45168/0 (socket says 172.21.15.8:45168) 2023-10-30T12:45:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ffa59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 0x7f1ffc1eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ffa59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 0x7f1ffc1eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55800/0 (socket says 172.21.15.8:55800) 2023-10-30T12:45:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ff9d9b640 1 -- 172.21.15.8:0/395675962 learned_addr learned my addr 172.21.15.8:0/395675962 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc1ea9f0 con 0x7f1ffc152a70 2023-10-30T12:45:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f1ffad9d640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 0x7f1ffc14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0c4360 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.137+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc150160 con 0x7f1ffc152e50 2023-10-30T12:45:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 -- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 msgr2=0x7f1ffc14fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 0x7f1ffc14fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 -- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 msgr2=0x7f1ffc1eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 0x7f1ffc1eae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0c4570 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ffad9d640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 0x7f1ffc14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:29.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1ff9d9b640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0010c60 tx=0x7f1ff000e540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff000ee50 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc1ea610 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0010ea0 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc14e900 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.141+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001c640 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.149+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0005260 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.149+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1ff001c7a0 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.149+0000 7f1feaffd640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 0x7f1fdc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.149+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ff0084510 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.153+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 0x7f1fdc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.157+0000 7f1ffa59c640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 0x7f1fdc068990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0af000 tx=0x7f1fec0046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.173+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1ff0051860 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.177+0000 7f1c0ddb5640 1 Processor -- start 2023-10-30T12:45:29.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.177+0000 7f1c0ddb5640 1 -- start start 2023-10-30T12:45:29.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0ddb5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c080ca0d0 0x7f1c081596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0ddb5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1c080c5560 0x7f1c08153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0ddb5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c080c4910 con 0x7f1c080ca4b0 2023-10-30T12:45:29.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0ddb5640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080b27b0 con 0x7f1c080c5560 2023-10-30T12:45:29.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0ddb5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080c4b10 con 0x7f1c080ca0d0 2023-10-30T12:45:29.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c080ca0d0 0x7f1c081596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c080ca0d0 0x7f1c081596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45180/0 (socket says 172.21.15.8:45180) 2023-10-30T12:45:29.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 -- 172.21.15.8:0/2376573549 learned_addr learned my addr 172.21.15.8:0/2376573549 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0d5b4640 1 --2- 172.21.15.8:0/2376573549 >> v2:172.21.15.8:3300/0 conn(0x7f1c080c5560 0x7f1c08153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1bff7fe640 1 -- 172.21.15.8:0/2376573549 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721273039 0 0) 0x7f1c080c4910 con 0x7f1c080ca4b0 2023-10-30T12:45:29.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1bff7fe640 1 -- 172.21.15.8:0/2376573549 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf0003530 con 0x7f1c080ca4b0 2023-10-30T12:45:29.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 -- 172.21.15.8:0/2376573549 >> v2:172.21.15.8:3300/0 conn(0x7f1c080c5560 msgr2=0x7f1c08153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 --2- 172.21.15.8:0/2376573549 >> v2:172.21.15.8:3300/0 conn(0x7f1c080c5560 0x7f1c08153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 -- 172.21.15.8:0/2376573549 >> v1:172.21.15.46:6789/0 conn(0x7f1c080ca4b0 legacy=0x7f1c080c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.181+0000 7f1c0cdb3640 1 -- 172.21.15.8:0/2376573549 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080c50d0 con 0x7f1c080ca0d0 2023-10-30T12:45:29.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.185+0000 7f87f1be4640 1 Processor -- start 2023-10-30T12:45:29.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.185+0000 7f87f1be4640 1 -- start start 2023-10-30T12:45:29.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.185+0000 7f87f1be4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87ec0bcea0 0x7f87ec159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.185+0000 7f87f1be4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec0c5460 0x7f87ec06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87ec153a20 0x7f87ec0bc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45092/0 (socket says 172.21.15.8:45092) 2023-10-30T12:45:29.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87ebfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87ec0bcea0 0x7f87ec159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 -- 172.21.15.8:0/4070125099 learned_addr learned my addr 172.21.15.8:0/4070125099 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:29.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec153e00 con 0x7f87ec153a20 2023-10-30T12:45:29.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f13e3640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec0c5460 0x7f87ec06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5aa0 con 0x7f87ec0bcea0 2023-10-30T12:45:29.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5d80 con 0x7f87ec0c5460 2023-10-30T12:45:29.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0cdb3640 1 --2- 172.21.15.8:0/2376573549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080ca0d0 0x7f1c081596f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c0806cc90 tx=0x7f1c0401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db23a44b15639199 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 Processor -- start 2023-10-30T12:45:29.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893550968 0 0) 0x7f87ec153e00 con 0x7f87ec153a20 2023-10-30T12:45:29.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d8003530 con 0x7f87ec153a20 2023-10-30T12:45:29.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1bff7fe640 1 -- 172.21.15.8:0/2376573549 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040aa430 con 0x7f1c080ca0d0 2023-10-30T12:45:29.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1bff7fe640 1 -- 172.21.15.8:0/2376573549 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c040aa590 con 0x7f1c080ca0d0 2023-10-30T12:45:29.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1bff7fe640 1 -- 172.21.15.8:0/2376573549 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040aa920 con 0x7f1c080ca0d0 2023-10-30T12:45:29.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 <== mon.1 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695962280 0 0) 0x7f87d8003530 con 0x7f87ec153a20 2023-10-30T12:45:29.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec0c5460 msgr2=0x7f87ec06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec0c5460 0x7f87ec06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 >> v2:172.21.15.46:3300/0 conn(0x7f87ec0bcea0 msgr2=0x7f87ec159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 --2- 172.21.15.8:0/4070125099 >> v2:172.21.15.46:3300/0 conn(0x7f87ec0bcea0 0x7f87ec159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0c6060 con 0x7f87ec153a20 2023-10-30T12:45:29.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 <== mon.1 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f87e4001680 con 0x7f87ec153a20 2023-10-30T12:45:29.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 >> v1:172.21.15.46:6789/0 conn(0x7f87ec153a20 legacy=0x7f87ec0bc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87d8003980 0x7f87d8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d80047d0 0x7f87d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87d8005190 0x7f87d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5aa0 con 0x7f87d80047d0 2023-10-30T12:45:29.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5d80 con 0x7f87d8003980 2023-10-30T12:45:29.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d800bf50 con 0x7f87d8005190 2023-10-30T12:45:29.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87d8003980 0x7f87d8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f13e3640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87d8005190 0x7f87d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87ebfff640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d80047d0 0x7f87d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/2376573549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080ca0d0 msgr2=0x7f1c081596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/2376573549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080ca0d0 0x7f1c081596f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c0806cc90 tx=0x7f1c0401c4d0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/2376573549 shutdown_connections 2023-10-30T12:45:29.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/2376573549 >> v2:172.21.15.8:3300/0 conn(0x7f1c080c5560 0x7f1c08153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/2376573549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080ca0d0 0x7f1c081596f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/2376573549 >> 172.21.15.8:0/2376573549 conn(0x7f1c0806bf20 msgr2=0x7f1c0806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/2376573549 shutdown_connections 2023-10-30T12:45:29.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/2376573549 wait complete. 2023-10-30T12:45:29.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7f1c0ddb5640 1 Processor -- start 2023-10-30T12:45:29.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7f1c0ddb5640 1 -- start start 2023-10-30T12:45:29.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 -- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87d8005190 msgr2=0x7f87d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87d8005190 0x7f87d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 -- 172.21.15.8:0/4070125099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d80047d0 msgr2=0x7f87d8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d80047d0 0x7f87d8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7f87f0be2640 1 -- 172.21.15.8:0/4070125099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8003530 con 0x7f87d8003980 2023-10-30T12:45:29.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 -- start start 2023-10-30T12:45:29.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7feb30153b10 0x7feb300c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feb300ca8e0 0x7feb3006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb30159900 con 0x7feb30153b10 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3015d070 con 0x7feb300ca8e0 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7feb3739d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3015d350 con 0x7feb3006c480 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7feb30153b10 0x7feb300c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7feb30153b10 0x7feb300c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55830/0 (socket says 172.21.15.8:55830) 2023-10-30T12:45:29.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 -- 172.21.15.8:0/141006084 learned_addr learned my addr 172.21.15.8:0/141006084 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 -- 172.21.15.8:0/141006084 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 msgr2=0x7feb30159d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:45:29.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 --2- 172.21.15.8:0/141006084 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30159d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 -- 172.21.15.8:0/141006084 >> v2:172.21.15.46:3300/0 conn(0x7feb300ca8e0 msgr2=0x7feb3006bf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-30T12:45:29.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 --2- 172.21.15.8:0/141006084 >> v2:172.21.15.46:3300/0 conn(0x7feb300ca8e0 0x7feb3006bf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7feb35139640 1 -- 172.21.15.8:0/141006084 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3015d630 con 0x7feb30153b10 2023-10-30T12:45:29.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7fa34711d640 1 Processor -- start 2023-10-30T12:45:29.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.189+0000 7fa34711d640 1 -- start start 2023-10-30T12:45:29.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa34711d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa34006bb40 0x7fa340157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa344eb9640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa34006b760 0x7fa34015e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53870/0 (socket says 172.21.15.8:53870) 2023-10-30T12:45:29.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa344eb9640 1 -- 172.21.15.8:0/1387170506 learned_addr learned my addr 172.21.15.8:0/1387170506 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:29.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa33ffff640 1 --2- 172.21.15.8:0/1387170506 >> v2:172.21.15.46:3300/0 conn(0x7fa34006bb40 0x7fa340157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340071e00 con 0x7fa34006b760 2023-10-30T12:45:29.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.193+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340070020 con 0x7fa34006c420 2023-10-30T12:45:29.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123436190 0 0) 0x7fa340071e00 con 0x7fa34006b760 2023-10-30T12:45:29.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080c5560 0x7f1c080b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c080ca0d0 0x7f1c081f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340070f10 con 0x7fa34006bb40 2023-10-30T12:45:29.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa330003530 con 0x7fa34006b760 2023-10-30T12:45:29.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593472236 0 0) 0x7fa340070020 con 0x7fa34006c420 2023-10-30T12:45:29.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa340071e00 con 0x7fa34006c420 2023-10-30T12:45:29.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08158660 con 0x7f1c080ca0d0 2023-10-30T12:45:29.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08145df0 con 0x7f1c080c5560 2023-10-30T12:45:29.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08146650 con 0x7f1c080ca4b0 2023-10-30T12:45:29.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f0be2640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87d8003980 0x7f87d8003da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c140 tx=0x7f87e401cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7024ee3cd097cfe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e40058e0 con 0x7f87d8003980 2023-10-30T12:45:29.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87eb7fe640 1 -- 172.21.15.8:0/4070125099 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4005a60 con 0x7f87d8003980 2023-10-30T12:45:29.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:51042/0 (socket says 172.21.15.8:51042) 2023-10-30T12:45:29.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 -- 172.21.15.8:0/1457511455 learned_addr learned my addr 172.21.15.8:0/1457511455 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0cdb3640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080c5560 0x7f1c080b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 -- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080c5560 msgr2=0x7f1c080b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080c5560 0x7f1c080b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 -- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c080ca0d0 msgr2=0x7f1c081f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c080ca0d0 0x7f1c081f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 -- 172.21.15.8:0/1457511455 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040094d0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0d5b4640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000efc0 tx=0x7f1c0000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33ffff640 1 -- 172.21.15.8:0/1387170506 >> v1:172.21.15.46:6789/0 conn(0x7fa34006c420 legacy=0x7fa3400c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33ffff640 1 -- 172.21.15.8:0/1387170506 >> v1:172.21.15.8:6789/0 conn(0x7fa34006b760 legacy=0x7fa34015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7fa33ffff640 1 -- 172.21.15.8:0/1387170506 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400711f0 con 0x7fa34006bb40 2023-10-30T12:45:29.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00014480 con 0x7f1c080ca4b0 2023-10-30T12:45:29.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87d8003980 msgr2=0x7f87d8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f1be4640 1 --2- 172.21.15.8:0/4070125099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87d8003980 0x7f87d8003da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c140 tx=0x7f87e401cba0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f13e3640 1 -- 172.21.15.8:0/4070125099 reap_dead start 2023-10-30T12:45:29.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081578c0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0000f040 con 0x7f1c080ca4b0 2023-10-30T12:45:29.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00014480 con 0x7f1c080ca4b0 2023-10-30T12:45:29.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c08146370 con 0x7f1c080ca4b0 2023-10-30T12:45:29.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7feb35139640 1 --2- 172.21.15.8:0/141006084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb30153b10 0x7feb300c9b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb300b1dd0 tx=0x7feb2001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108f413e17ccad9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 shutdown_connections 2023-10-30T12:45:29.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 >> 172.21.15.8:0/4070125099 conn(0x7f87ec0c7560 msgr2=0x7f87ec0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f1129417640 1 Processor -- start 2023-10-30T12:45:29.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.197+0000 7f8191b72640 1 Processor -- start 2023-10-30T12:45:29.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 -- start start 2023-10-30T12:45:29.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c081f5ba0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb27fff640 1 -- 172.21.15.8:0/141006084 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200ab610 con 0x7feb30153b10 2023-10-30T12:45:29.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 shutdown_connections 2023-10-30T12:45:29.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa33ffff640 1 --2- 172.21.15.8:0/1387170506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006bb40 0x7fa340157a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0097c0 tx=0x7fa32c01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291ad8c31eaf1ed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c000145e0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb27fff640 1 -- 172.21.15.8:0/141006084 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200ab770 con 0x7feb30153b10 2023-10-30T12:45:29.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb27fff640 1 -- 172.21.15.8:0/141006084 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200abb00 con 0x7feb30153b10 2023-10-30T12:45:29.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 -- 172.21.15.8:0/141006084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb30153b10 msgr2=0x7feb300c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 --2- 172.21.15.8:0/141006084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb30153b10 0x7feb300c9b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb300b1dd0 tx=0x7feb2001c540 comp rx=0 tx=0).stop 2023-10-30T12:45:29.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 0x7f112406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f11240c7110 0x7f11240c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f112406b500 0x7f112406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0ab650 con 0x7fa34006bb40 2023-10-30T12:45:29.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c0ab7b0 con 0x7fa34006bb40 2023-10-30T12:45:29.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 -- start start 2023-10-30T12:45:29.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa33f7fe640 1 -- 172.21.15.8:0/1387170506 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0abb40 con 0x7fa34006bb40 2023-10-30T12:45:29.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 0x7f818c0bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 0x7f818c0c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c06bcf0 con 0x7f818c0c67a0 2023-10-30T12:45:29.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0c5ec0 con 0x7f818c15e5d0 2023-10-30T12:45:29.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f8191b72640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15ea60 con 0x7f818c157a70 2023-10-30T12:45:29.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f11237fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f112406b500 0x7f112406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f11237fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f112406b500 0x7f112406b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45214/0 (socket says 172.21.15.8:45214) 2023-10-30T12:45:29.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f11237fe640 1 -- 172.21.15.8:0/1275048573 learned_addr learned my addr 172.21.15.8:0/1275048573 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 -- 172.21.15.8:0/141006084 shutdown_connections 2023-10-30T12:45:29.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 --2- 172.21.15.8:0/141006084 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 --2- 172.21.15.8:0/141006084 >> v2:172.21.15.46:3300/0 conn(0x7feb300ca8e0 0x7feb3006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 --2- 172.21.15.8:0/141006084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb30153b10 0x7feb300c9b90 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb300b1dd0 tx=0x7feb2001c540 comp rx=0 tx=0).stop 2023-10-30T12:45:29.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7feb3739d640 1 -- 172.21.15.8:0/141006084 >> 172.21.15.8:0/141006084 conn(0x7feb3006a240 msgr2=0x7feb3006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006bb40 msgr2=0x7fa340157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7fa34711d640 1 --2- 172.21.15.8:0/1387170506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006bb40 0x7fa340157a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0097c0 tx=0x7fa32c01c490 comp rx=0 tx=0).stop 2023-10-30T12:45:29.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240c6030 con 0x7f112414fb70 2023-10-30T12:45:29.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112414ff50 con 0x7f112406b500 2023-10-30T12:45:29.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240c6d10 con 0x7f11240c7110 2023-10-30T12:45:29.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f818affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f818c0c67a0 0x7f818c06c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45106/0 (socket says 172.21.15.8:45106) 2023-10-30T12:45:29.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818affd640 1 -- 172.21.15.8:0/2181218488 learned_addr learned my addr 172.21.15.8:0/2181218488 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:29.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f11227fc640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f11240c7110 0x7f11240c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f1122ffd640 1 --2- 172.21.15.8:0/1275048573 >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 0x7f112406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.201+0000 7f87f1be4640 1 -- 172.21.15.8:0/4070125099 wait complete. 2023-10-30T12:45:29.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f87f1be4640 1 Processor -- start 2023-10-30T12:45:29.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 shutdown_connections 2023-10-30T12:45:29.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7fa34711d640 1 --2- 172.21.15.8:0/1387170506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006bb40 0x7fa340157a20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 >> 172.21.15.8:0/1387170506 conn(0x7fa3400cac70 msgr2=0x7fa3400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f1bfd7fa640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 0x7f1bf40689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 shutdown_connections 2023-10-30T12:45:29.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818a7fc640 1 -- 172.21.15.8:0/2181218488 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067261996 0 0) 0x7f818c06bcf0 con 0x7f818c0c67a0 2023-10-30T12:45:29.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f87f1be4640 1 -- start start 2023-10-30T12:45:29.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c00084a10 con 0x7f1c080ca4b0 2023-10-30T12:45:29.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7feb3739d640 1 -- 172.21.15.8:0/141006084 shutdown_connections 2023-10-30T12:45:29.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818b7fe640 1 --2- 172.21.15.8:0/2181218488 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 0x7f818c0bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f11237fe640 1 -- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f11240c7110 msgr2=0x7f11240c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f11237fe640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f11240c7110 0x7f11240c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f11237fe640 1 -- 172.21.15.8:0/1275048573 >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 msgr2=0x7f112406fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f11237fe640 1 --2- 172.21.15.8:0/1275048573 >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 0x7f112406fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f11237fe640 1 -- 172.21.15.8:0/1275048573 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11241563b0 con 0x7f112406b500 2023-10-30T12:45:29.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f1122ffd640 1 --2- 172.21.15.8:0/1275048573 >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 0x7f112406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:29.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818b7fe640 1 -- 172.21.15.8:0/2181218488 >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 msgr2=0x7f818c0c05d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-30T12:45:29.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7feb3739d640 1 -- 172.21.15.8:0/141006084 wait complete. 2023-10-30T12:45:29.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7fa34711d640 1 -- 172.21.15.8:0/1387170506 wait complete. 2023-10-30T12:45:29.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818bfff640 1 --2- 172.21.15.8:0/2181218488 >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 0x7f818c0c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f1c0cdb3640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 0x7f1bf40689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f87f1be4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 0x7f87ec152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7feb3739d640 1 Processor -- start 2023-10-30T12:45:29.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 Processor -- start 2023-10-30T12:45:29.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818b7fe640 1 --2- 172.21.15.8:0/2181218488 >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 0x7f818c0c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818b7fe640 1 -- 172.21.15.8:0/2181218488 >> v1:172.21.15.46:6789/0 conn(0x7f818c0c67a0 legacy=0x7f818c06c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818b7fe640 1 -- 172.21.15.8:0/2181218488 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0c5680 con 0x7f818c157a70 2023-10-30T12:45:29.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f87f0be2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 0x7f87ec152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.205+0000 7f818bfff640 1 --2- 172.21.15.8:0/2181218488 >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 0x7f818c0c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:29.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 -- start start 2023-10-30T12:45:29.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f1be4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 0x7f87ec1fe940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f0be2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 0x7f87ec152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55866/0 (socket says 172.21.15.8:55866) 2023-10-30T12:45:29.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f1be4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec070560 0x7f87ec070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f1c0cdb3640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 0x7f1bf40689c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c040ae000 tx=0x7f1c040a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 -- start start 2023-10-30T12:45:29.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f1be4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec154f30 con 0x7f87d8003980 2023-10-30T12:45:29.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec070f50 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 0x7f87ec1fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1fee80 con 0x7f87ec070560 2023-10-30T12:45:29.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f0be2640 1 -- 172.21.15.8:0/1404505536 learned_addr learned my addr 172.21.15.8:0/1404505536 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87f13e3640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec070560 0x7f87ec070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c081f5ba0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 -- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec070560 msgr2=0x7f87ec070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec070560 0x7f87ec070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 -- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 msgr2=0x7f87ec152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 0x7f87ec152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f87ebfff640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec1540d0 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb30147f50 0x7feb30143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb301447a0 con 0x7feb300ca8e0 2023-10-30T12:45:29.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa34006b760 0x7fa3400c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 0x7fa3401f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa340150d40 con 0x7fa34006bb40 2023-10-30T12:45:29.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400c3f00 con 0x7fa34006c420 2023-10-30T12:45:29.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa34711d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400c47d0 con 0x7fa34006b760 2023-10-30T12:45:29.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb30152d70 con 0x7feb30147f50 2023-10-30T12:45:29.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3739d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3015ad50 con 0x7feb3006c480 2023-10-30T12:45:29.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55868/0 (socket says 172.21.15.8:55868) 2023-10-30T12:45:29.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 -- 172.21.15.8:0/1523447954 learned_addr learned my addr 172.21.15.8:0/1523447954 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb3593a640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb30147f50 0x7feb30143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa33ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 0x7fa3401f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa3456ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa344eb9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa34006b760 0x7fa3400c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa3456ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45248/0 (socket says 172.21.15.8:45248) 2023-10-30T12:45:29.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa3456ba640 1 -- 172.21.15.8:0/3370527246 learned_addr learned my addr 172.21.15.8:0/3370527246 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7fa33ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 0x7fa3401f9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55872/0 (socket says 172.21.15.8:55872) 2023-10-30T12:45:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 -- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 msgr2=0x7feb30146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 -- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb30147f50 msgr2=0x7feb30143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb30147f50 0x7feb30143290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7feb34938640 1 -- 172.21.15.8:0/1523447954 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200094d0 con 0x7feb300ca8e0 2023-10-30T12:45:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f11237fe640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112406b500 0x7f112406b930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f112406d080 tx=0x7f111c020870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19859bddbf620401 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f1121ffb640 1 -- 172.21.15.8:0/1275048573 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c008be0 con 0x7f112406b500 2023-10-30T12:45:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f1121ffb640 1 -- 172.21.15.8:0/1275048573 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c008d40 con 0x7f112406b500 2023-10-30T12:45:29.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.209+0000 7f1121ffb640 1 -- 172.21.15.8:0/1275048573 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00a2f0 con 0x7f112406b500 2023-10-30T12:45:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87ebfff640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 0x7f87ec1fe940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f87dc011990 tx=0x7f87dc011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112406b500 msgr2=0x7f112406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112406b500 0x7f112406b930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f112406d080 tx=0x7f111c020870 comp rx=0 tx=0).stop 2023-10-30T12:45:29.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb34938640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b410 tx=0x7feb2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004300 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc004460 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00faa0 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 shutdown_connections 2023-10-30T12:45:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112406b500 0x7f112406b930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 --2- 172.21.15.8:0/1275048573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f11240c7110 0x7f11240c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 --2- 172.21.15.8:0/1275048573 >> v2:172.21.15.8:3300/0 conn(0x7f112414fb70 0x7f112406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 >> 172.21.15.8:0/1275048573 conn(0x7f11240cac70 msgr2=0x7f11240cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1543b0 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec154970 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c011830 con 0x7feb300ca8e0 2023-10-30T12:45:29.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c011990 con 0x7feb300ca8e0 2023-10-30T12:45:29.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c011d40 con 0x7feb300ca8e0 2023-10-30T12:45:29.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 -- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa34006b760 msgr2=0x7fa3400c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa34006b760 0x7fa3400c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 -- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 msgr2=0x7fa3401f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 0x7fa3401f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 -- 172.21.15.8:0/3370527246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c009420 con 0x7fa34006c420 2023-10-30T12:45:29.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f818b7fe640 1 --2- 172.21.15.8:0/2181218488 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 0x7f818c0bfbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8184009d90 tx=0x7f818401f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ecc1fc47b4a1a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 shutdown_connections 2023-10-30T12:45:29.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb301523a0 con 0x7feb300ca8e0 2023-10-30T12:45:29.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f818a7fc640 1 -- 172.21.15.8:0/2181218488 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840af600 con 0x7f818c157a70 2023-10-30T12:45:29.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f818a7fc640 1 -- 172.21.15.8:0/2181218488 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840ab050 con 0x7f818c157a70 2023-10-30T12:45:29.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f818a7fc640 1 -- 172.21.15.8:0/2181218488 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840af600 con 0x7f818c157a70 2023-10-30T12:45:29.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 -- 172.21.15.8:0/2181218488 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 msgr2=0x7f818c0bfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 --2- 172.21.15.8:0/2181218488 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 0x7f818c0bfbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8184009d90 tx=0x7f818401f130 comp rx=0 tx=0).stop 2023-10-30T12:45:29.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec04e170 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f1129417640 1 -- 172.21.15.8:0/1275048573 wait complete. 2023-10-30T12:45:29.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 -- 172.21.15.8:0/2181218488 shutdown_connections 2023-10-30T12:45:29.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 --2- 172.21.15.8:0/2181218488 >> v2:172.21.15.8:3300/0 conn(0x7f818c15e5d0 0x7f818c0c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 --2- 172.21.15.8:0/2181218488 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c157a70 0x7f818c0bfbc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 -- 172.21.15.8:0/2181218488 >> 172.21.15.8:0/2181218488 conn(0x7f818c069e90 msgr2=0x7f818c06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa3456ba640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa33800c730 tx=0x7fa33800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7f8191b72640 1 -- 172.21.15.8:0/2181218488 shutdown_connections 2023-10-30T12:45:29.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb30144c90 con 0x7feb300ca8e0 2023-10-30T12:45:29.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338003b80 con 0x7fa34006c420 2023-10-30T12:45:29.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb30067540 con 0x7feb300ca8e0 2023-10-30T12:45:29.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa338003ce0 con 0x7fa34006c420 2023-10-30T12:45:29.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.213+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800aa90 con 0x7fa34006c420 2023-10-30T12:45:29.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34015ec70 con 0x7fa34006c420 2023-10-30T12:45:29.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3400c44f0 con 0x7fa34006c420 2023-10-30T12:45:29.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1129417640 1 Processor -- start 2023-10-30T12:45:29.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87dc00fc00 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1129417640 1 -- start start 2023-10-30T12:45:29.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb2c01a410 con 0x7feb300ca8e0 2023-10-30T12:45:29.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f87e97fa640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 0x7f87cc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87dc0843c0 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f8191b72640 1 -- 172.21.15.8:0/2181218488 wait complete. 2023-10-30T12:45:29.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f8191b72640 1 Processor -- start 2023-10-30T12:45:29.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1129417640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 0x7f11241462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1129417640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 0x7f11241f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1129417640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f8191b72640 1 -- start start 2023-10-30T12:45:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1122ffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 0x7f11241462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.217+0000 7f1122ffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 0x7f11241462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:51102/0 (socket says 172.21.15.8:51102) 2023-10-30T12:45:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7feb25ffb640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 0x7feb140689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11227fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 0x7f11241f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34004e170 con 0x7fa34006c420 2023-10-30T12:45:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11227fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 0x7f11241f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45258/0 (socket says 172.21.15.8:45258) 2023-10-30T12:45:29.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f87f0be2640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 0x7f87cc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55888/0 (socket says 172.21.15.8:55888) 2023-10-30T12:45:29.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa33800abf0 con 0x7fa34006c420 2023-10-30T12:45:29.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f1129417640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11241eef90 con 0x7f112414fb70 2023-10-30T12:45:29.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f1129417640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11241f1510 con 0x7f11240c7110 2023-10-30T12:45:29.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f818c0c67a0 0x7f818c154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f1129417640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11241f17f0 con 0x7f112406b500 2023-10-30T12:45:29.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 0x7f818c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f1122ffd640 1 -- 172.21.15.8:0/510044197 learned_addr learned my addr 172.21.15.8:0/510044197 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45262/0 (socket says 172.21.15.8:45262) 2023-10-30T12:45:29.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818affd640 1 -- 172.21.15.8:0/292459902 learned_addr learned my addr 172.21.15.8:0/292459902 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb2c083150 con 0x7feb300ca8e0 2023-10-30T12:45:29.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7feb35139640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 0x7feb140689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c071d20 con 0x7f818c0c67a0 2023-10-30T12:45:29.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818bfff640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 0x7f818c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c151e90 con 0x7f818c157a70 2023-10-30T12:45:29.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c071fc0 con 0x7f818c15e5d0 2023-10-30T12:45:29.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7fa33d7fa640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 0x7fa3240689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f87f0be2640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 0x7f87cc068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87e40ab040 tx=0x7f87e40019b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7fa344eb9640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 0x7fa3240689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818bfff640 1 -- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 msgr2=0x7f818c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 -- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 msgr2=0x7f11241462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 0x7f11241462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 -- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 msgr2=0x7f11241f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 0x7f11241f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f11237fe640 1 -- 172.21.15.8:0/510044197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c007040 con 0x7f112414fb70 2023-10-30T12:45:29.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.221+0000 7f818bfff640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f818bfff640 1 -- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f818c0c67a0 msgr2=0x7f818c154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f818bfff640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f818c0c67a0 0x7f818c154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f818bfff640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184009a70 con 0x7f818c15e5d0 2023-10-30T12:45:29.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f818affd640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:29.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f11237fe640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f111c0af000 tx=0x7f111c01d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f818bfff640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 0x7f818c153980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f818000c5e0 tx=0x7f818000caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7fa344eb9640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 0x7fa3240689b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f70 tx=0x7fa334007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011800 con 0x7f818c15e5d0 2023-10-30T12:45:29.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81800119a0 con 0x7f818c15e5d0 2023-10-30T12:45:29.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011d50 con 0x7f818c15e5d0 2023-10-30T12:45:29.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c0cb550 con 0x7f818c15e5d0 2023-10-30T12:45:29.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c0c5190 con 0x7f818c15e5d0 2023-10-30T12:45:29.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa33800ce40 con 0x7fa34006c420 2023-10-30T12:45:29.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87dc0517c0 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7feb35139640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 0x7feb140689e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb200af000 tx=0x7feb20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00a8e0 con 0x7f112414fb70 2023-10-30T12:45:29.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.225+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1124146810 con 0x7f112414fb70 2023-10-30T12:45:29.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1124148ae0 con 0x7f112414fb70 2023-10-30T12:45:29.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c013030 con 0x7f112414fb70 2023-10-30T12:45:29.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c013400 con 0x7f112414fb70 2023-10-30T12:45:29.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c0cb3b0 con 0x7f818c15e5d0 2023-10-30T12:45:29.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb2c088060 con 0x7feb300ca8e0 2023-10-30T12:45:29.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.229+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8180022350 con 0x7f818c15e5d0 2023-10-30T12:45:29.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f8173fff640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 0x7f8178068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1108005260 con 0x7f112414fb70 2023-10-30T12:45:29.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f111c0a9590 con 0x7f112414fb70 2023-10-30T12:45:29.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8180083eb0 con 0x7f818c15e5d0 2023-10-30T12:45:29.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f818b7fe640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 0x7f8178068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f111b7fe640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 0x7f11040689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f111c11e710 con 0x7f112414fb70 2023-10-30T12:45:29.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f1122ffd640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 0x7f11040689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.233+0000 7f818b7fe640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 0x7f8178068a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81840b1000 tx=0x7f8184004ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.237+0000 7f1122ffd640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 0x7f11040689b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f110c00acd0 tx=0x7f110c004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.237+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa338051070 con 0x7fa34006c420 2023-10-30T12:45:29.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.241+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f818c0cb3b0 con 0x7f818c15e5d0 2023-10-30T12:45:29.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.241+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f111c0ebc90 con 0x7f112414fb70 2023-10-30T12:45:29.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.461+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f86c8067540 con 0x7f86c80cf4e0 2023-10-30T12:45:29.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.465+0000 7f86be7fc640 1 -- 172.21.15.8:0/2740237304 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86b8051310 con 0x7f86c80cf4e0 2023-10-30T12:45:29.467 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:29.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 msgr2=0x7f86ac0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 0x7f86ac0689e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b2000 tx=0x7f86c4008040 comp rx=0 tx=0).stop 2023-10-30T12:45:29.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 msgr2=0x7f86c80b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86b800b550 tx=0x7f86b800ba10 comp rx=0 tx=0).stop 2023-10-30T12:45:29.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 shutdown_connections 2023-10-30T12:45:29.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86ac066520 0x7f86ac0689e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c80c8c40 0x7f86c80c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86c80cf4e0 0x7f86c80b2a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 --2- 172.21.15.8:0/2740237304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f86c80c9ef0 0x7f86c81f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 >> 172.21.15.8:0/2740237304 conn(0x7f86c806bf60 msgr2=0x7f86c806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 shutdown_connections 2023-10-30T12:45:29.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.469+0000 7f86d0009640 1 -- 172.21.15.8:0/2740237304 wait complete. 2023-10-30T12:45:29.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-30T12:45:29.503 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.505+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1fe0005510 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.505+0000 7f1feaffd640 1 -- 172.21.15.8:0/395675962 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ff0054e80 con 0x7f1ffc0c20b0 2023-10-30T12:45:29.507 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:29.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 msgr2=0x7f1fdc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 0x7f1fdc068990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0af000 tx=0x7f1fec0046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 msgr2=0x7f1ffc14f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0010c60 tx=0x7f1ff000e540 comp rx=0 tx=0).stop 2023-10-30T12:45:29.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 shutdown_connections 2023-10-30T12:45:29.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1fdc0664d0 0x7f1fdc068990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ffc0c20b0 0x7f1ffc14f6e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1ffc152e50 0x7f1ffc14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 --2- 172.21.15.8:0/395675962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ffc152a70 0x7f1ffc1eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 >> 172.21.15.8:0/395675962 conn(0x7f1ffc06d810 msgr2=0x7f1ffc06fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 shutdown_connections 2023-10-30T12:45:29.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.509+0000 7f2000c46640 1 -- 172.21.15.8:0/395675962 wait complete. 2023-10-30T12:45:29.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.537+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f818c0cb3b0 con 0x7f818c15e5d0 2023-10-30T12:45:29.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.537+0000 7f8173fff640 1 -- 172.21.15.8:0/292459902 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f818c0cb3b0 con 0x7f818c15e5d0 2023-10-30T12:45:29.542 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:29.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-30T12:45:29.543 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 msgr2=0x7f8178068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 0x7f8178068a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81840b1000 tx=0x7f8184004ac0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 msgr2=0x7f818c153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 0x7f818c153980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f818000c5e0 tx=0x7f818000caa0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 shutdown_connections 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81780665a0 0x7f8178068a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818c157a70 0x7f818c0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f818c15e5d0 0x7f818c153980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 --2- 172.21.15.8:0/292459902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f818c0c67a0 0x7f818c154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 >> 172.21.15.8:0/292459902 conn(0x7f818c069e90 msgr2=0x7f818c0c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 shutdown_connections 2023-10-30T12:45:29.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.541+0000 7f8191b72640 1 -- 172.21.15.8:0/292459902 wait complete. 2023-10-30T12:45:29.549 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:29.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.545+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1c081f5ba0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.545+0000 7f1bfd7fa640 1 -- 172.21.15.8:0/1457511455 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c081f5ba0 con 0x7f1c080ca4b0 2023-10-30T12:45:29.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 msgr2=0x7f1bf40689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 0x7f1bf40689c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c040ae000 tx=0x7f1c040a7000 comp rx=0 tx=0).stop 2023-10-30T12:45:29.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 msgr2=0x7f1c081f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000efc0 tx=0x7f1c0000c490 comp rx=0 tx=0).stop 2023-10-30T12:45:29.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 shutdown_connections 2023-10-30T12:45:29.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bf4066500 0x7f1bf40689c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1c080ca4b0 0x7f1c081f63c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c080ca0d0 0x7f1c081f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 --2- 172.21.15.8:0/1457511455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c080c5560 0x7f1c080b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.549+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 >> 172.21.15.8:0/1457511455 conn(0x7f1c0806bf20 msgr2=0x7f1c08150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.553+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 shutdown_connections 2023-10-30T12:45:29.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.553+0000 7f1c0ddb5640 1 -- 172.21.15.8:0/1457511455 wait complete. 2023-10-30T12:45:29.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.553+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f87ec06e560 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.553+0000 7f87e97fa640 1 -- 172.21.15.8:0/1404505536 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87dc019090 con 0x7f87ec1fe4e0 2023-10-30T12:45:29.558 INFO:teuthology.orchestra.run.smithi008.stdout:55834574856 2023-10-30T12:45:29.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 msgr2=0x7f87cc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 0x7f87cc068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87e40ab040 tx=0x7f87e40019b0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 msgr2=0x7f87ec1fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 0x7f87ec1fe940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f87dc011990 tx=0x7f87dc011e50 comp rx=0 tx=0).stop 2023-10-30T12:45:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 shutdown_connections 2023-10-30T12:45:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f87cc0664d0 0x7f87cc068990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec1fe4e0 0x7f87ec1fe940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f87ec070560 0x7f87ec070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 --2- 172.21.15.8:0/1404505536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8003980 0x7f87ec152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.557+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 >> 172.21.15.8:0/1404505536 conn(0x7f87ec0c7560 msgr2=0x7f87ec06d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 shutdown_connections 2023-10-30T12:45:29.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f87f1be4640 1 -- 172.21.15.8:0/1404505536 wait complete. 2023-10-30T12:45:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11080057e0 con 0x7f112414fb70 2023-10-30T12:45:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f111b7fe640 1 -- 172.21.15.8:0/510044197 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f111c0ef2b0 con 0x7f112414fb70 2023-10-30T12:45:29.564 INFO:teuthology.orchestra.run.smithi008.stdout:55834574854 2023-10-30T12:45:29.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 msgr2=0x7f11040689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 0x7f11040689b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f110c00acd0 tx=0x7f110c004740 comp rx=0 tx=0).stop 2023-10-30T12:45:29.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 msgr2=0x7f112414ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f111c0af000 tx=0x7f111c01d610 comp rx=0 tx=0).stop 2023-10-30T12:45:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 shutdown_connections 2023-10-30T12:45:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f11040664f0 0x7f11040689b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11240c7110 0x7f11241f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f112414fb70 0x7f112414ea20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 --2- 172.21.15.8:0/510044197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f112406b500 0x7f11241462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 >> 172.21.15.8:0/510044197 conn(0x7f11240cac70 msgr2=0x7f11241494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 shutdown_connections 2023-10-30T12:45:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.561+0000 7f1129417640 1 -- 172.21.15.8:0/510044197 wait complete. 2023-10-30T12:45:29.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-30T12:45:29.570 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.569+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7feb30152530 con 0x7feb300ca8e0 2023-10-30T12:45:29.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.569+0000 7feb25ffb640 1 -- 172.21.15.8:0/1523447954 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb2c050530 con 0x7feb300ca8e0 2023-10-30T12:45:29.573 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:29.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 msgr2=0x7feb140689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 0x7feb140689e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb200af000 tx=0x7feb20008040 comp rx=0 tx=0).stop 2023-10-30T12:45:29.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 msgr2=0x7feb30152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b410 tx=0x7feb2c00b8d0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 shutdown_connections 2023-10-30T12:45:29.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feb14066520 0x7feb140689e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb30147f50 0x7feb30143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb300ca8e0 0x7feb30152830 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 --2- 172.21.15.8:0/1523447954 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7feb3006c480 0x7feb30146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 >> 172.21.15.8:0/1523447954 conn(0x7feb3006a240 msgr2=0x7feb3006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 shutdown_connections 2023-10-30T12:45:29.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.573+0000 7feb3739d640 1 -- 172.21.15.8:0/1523447954 wait complete. 2023-10-30T12:45:29.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.577+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa34014da10 con 0x7fa34006c420 2023-10-30T12:45:29.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.577+0000 7fa33d7fa640 1 -- 172.21.15.8:0/3370527246 <== mon.1 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa338054690 con 0x7fa34006c420 2023-10-30T12:45:29.580 INFO:teuthology.orchestra.run.smithi008.stdout:55834574855 2023-10-30T12:45:29.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 msgr2=0x7fa3240689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 0x7fa3240689b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f70 tx=0x7fa334007d40 comp rx=0 tx=0).stop 2023-10-30T12:45:29.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 msgr2=0x7fa34014b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa33800c730 tx=0x7fa33800cbf0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 shutdown_connections 2023-10-30T12:45:29.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa3240664f0 0x7fa3240689b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa34006c420 0x7fa34014b740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa34006bb40 0x7fa3401f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 --2- 172.21.15.8:0/3370527246 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa34006b760 0x7fa3400c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 >> 172.21.15.8:0/3370527246 conn(0x7fa3400cac70 msgr2=0x7fa3400c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 shutdown_connections 2023-10-30T12:45:29.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.581+0000 7fa34711d640 1 -- 172.21.15.8:0/3370527246 wait complete. 2023-10-30T12:45:29.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-30T12:45:29.588 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-30T12:45:29.596 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-30T12:45:29.597 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-30T12:45:29.603 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-30T12:45:29.607 DEBUG:teuthology.parallel:result is None 2023-10-30T12:45:29.607 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T12:45:29.608 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:29.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 Processor -- start 2023-10-30T12:45:29.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 -- start start 2023-10-30T12:45:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa4280c4a10 0x7fa42815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4280c42a0 0x7fa4280ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa428159f20 con 0x7fa428153c80 2023-10-30T12:45:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42815dec0 con 0x7fa4280c4a10 2023-10-30T12:45:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42f701640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4281599a0 con 0x7fa4280c42a0 2023-10-30T12:45:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42cc9c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa4280c4a10 0x7fa42815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42d49d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa428153c80 0x7fa42815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53874/0 (socket says 172.21.15.8:53874) 2023-10-30T12:45:29.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42d49d640 1 -- 172.21.15.8:0/1172983456 learned_addr learned my addr 172.21.15.8:0/1172983456 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:29.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42dc9e640 1 --2- 172.21.15.8:0/1172983456 >> v2:172.21.15.46:3300/0 conn(0x7fa4280c42a0 0x7fa4280ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42cc9c640 1 -- 172.21.15.8:0/1172983456 >> v2:172.21.15.46:3300/0 conn(0x7fa4280c42a0 msgr2=0x7fa4280ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:29.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42cc9c640 1 --2- 172.21.15.8:0/1172983456 >> v2:172.21.15.46:3300/0 conn(0x7fa4280c42a0 0x7fa4280ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42cc9c640 1 -- 172.21.15.8:0/1172983456 >> v1:172.21.15.8:6789/0 conn(0x7fa428153c80 legacy=0x7fa42815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa42cc9c640 1 -- 172.21.15.8:0/1172983456 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42815d620 con 0x7fa4280c4a10 2023-10-30T12:45:29.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.793+0000 7fa41ffff640 1 -- 172.21.15.8:0/1172983456 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283576879 0 0) 0x7fa428159f20 con 0x7fa428153c80 2023-10-30T12:45:29.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42cc9c640 1 --2- 172.21.15.8:0/1172983456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4280c4a10 0x7fa42815cf90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa4280b1d40 tx=0x7fa41801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aace2582dce1ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa41ffff640 1 -- 172.21.15.8:0/1172983456 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180a8950 con 0x7fa4280c4a10 2023-10-30T12:45:29.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa41ffff640 1 -- 172.21.15.8:0/1172983456 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4180a8ab0 con 0x7fa4280c4a10 2023-10-30T12:45:29.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa41ffff640 1 -- 172.21.15.8:0/1172983456 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180a8e40 con 0x7fa4280c4a10 2023-10-30T12:45:29.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 -- 172.21.15.8:0/1172983456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4280c4a10 msgr2=0x7fa42815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 --2- 172.21.15.8:0/1172983456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4280c4a10 0x7fa42815cf90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa4280b1d40 tx=0x7fa41801ec50 comp rx=0 tx=0).stop 2023-10-30T12:45:29.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 -- 172.21.15.8:0/1172983456 shutdown_connections 2023-10-30T12:45:29.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 --2- 172.21.15.8:0/1172983456 >> v2:172.21.15.46:3300/0 conn(0x7fa4280c42a0 0x7fa4280ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 --2- 172.21.15.8:0/1172983456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4280c4a10 0x7fa42815cf90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 -- 172.21.15.8:0/1172983456 >> 172.21.15.8:0/1172983456 conn(0x7fa4280715c0 msgr2=0x7fa4280719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 -- 172.21.15.8:0/1172983456 shutdown_connections 2023-10-30T12:45:29.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.797+0000 7fa42f701640 1 -- 172.21.15.8:0/1172983456 wait complete. 2023-10-30T12:45:29.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 Processor -- start 2023-10-30T12:45:29.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- start start 2023-10-30T12:45:29.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa4280c42a0 0x7fa428155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 0x7fa4281f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42814ea10 con 0x7fa428153c80 2023-10-30T12:45:29.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42814ded0 con 0x7fa4280c4a10 2023-10-30T12:45:29.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281541e0 con 0x7fa4280c42a0 2023-10-30T12:45:29.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55912/0 (socket says 172.21.15.8:55912) 2023-10-30T12:45:29.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 -- 172.21.15.8:0/119456736 learned_addr learned my addr 172.21.15.8:0/119456736 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:29.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42cc9c640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 0x7fa4281f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42d49d640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa4280c42a0 0x7fa428155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 -- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa4280c42a0 msgr2=0x7fa428155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa4280c42a0 0x7fa428155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 -- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 msgr2=0x7fa4281f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 0x7fa4281f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 -- 172.21.15.8:0/119456736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009420 con 0x7fa428153c80 2023-10-30T12:45:29.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42cc9c640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 0x7fa4281f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:29.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42dc9e640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e40 tx=0x7fa42000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420014620 con 0x7fa428153c80 2023-10-30T12:45:29.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42000ee90 con 0x7fa428153c80 2023-10-30T12:45:29.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42814e0e0 con 0x7fa428153c80 2023-10-30T12:45:29.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200149e0 con 0x7fa428153c80 2023-10-30T12:45:29.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.801+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42814e6a0 con 0x7fa428153c80 2023-10-30T12:45:29.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42804e170 con 0x7fa428153c80 2023-10-30T12:45:29.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa420014b40 con 0x7fa428153c80 2023-10-30T12:45:29.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa41dffb640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 0x7fa40c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:29.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa420084830 con 0x7fa428153c80 2023-10-30T12:45:29.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa42d49d640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 0x7fa40c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:29.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.805+0000 7fa42d49d640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 0x7fa40c068990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f70 tx=0x7fa424007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:29.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.809+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa420051ca0 con 0x7fa428153c80 2023-10-30T12:45:29.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.965+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa42815dbe0 con 0x7fa40c0664d0 2023-10-30T12:45:29.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.969+0000 7fa41dffb640 1 -- 172.21.15.8:0/119456736 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264535 (secure 0 0 0) 0x7fa424007da0 con 0x7fa40c0664d0 2023-10-30T12:45:29.976 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:29.978 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:29.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 msgr2=0x7fa40c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 0x7fa40c068990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f70 tx=0x7fa424007c10 comp rx=0 tx=0).stop 2023-10-30T12:45:29.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 msgr2=0x7fa4281f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:29.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa420007e40 tx=0x7fa42000e4b0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 shutdown_connections 2023-10-30T12:45:29.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa40c0664d0 0x7fa40c068990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4280c4a10 0x7fa4281f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa428153c80 0x7fa4281f2960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 --2- 172.21.15.8:0/119456736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa4280c42a0 0x7fa428155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:29.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 >> 172.21.15.8:0/119456736 conn(0x7fa4280715c0 msgr2=0x7fa4281587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:29.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 shutdown_connections 2023-10-30T12:45:29.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:29.973+0000 7fa42f701640 1 -- 172.21.15.8:0/119456736 wait complete. 2023-10-30T12:45:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-30T12:45:29.614691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252228,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722492,"statfs":{"total":773094113280,"available":772835831808,"internally_reserved":0,"allocated":2887680,"data_stored":1751721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005823"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071955+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:20.071955+0000","last_peered":"2023-10-30T12:45:20.071955+0000","last_clean":"2023-10-30T12:45:20.071955+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:20.071955+0000","last_undegraded":"2023-10-30T12:45:20.071955+0000","last_fullsized":"2023-10-30T12:45:20.071955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:33:48.910689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067349+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:20.067349+0000","last_peered":"2023-10-30T12:45:20.067349+0000","last_clean":"2023-10-30T12:45:20.067349+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:20.067349+0000","last_undegraded":"2023-10-30T12:45:20.067349+0000","last_fullsized":"2023-10-30T12:45:20.067349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:34.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470964+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:20.470964+0000","last_peered":"2023-10-30T12:45:20.470964+0000","last_clean":"2023-10-30T12:45:20.470964+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:20.470964+0000","last_undegraded":"2023-10-30T12:45:20.470964+0000","last_fullsized":"2023-10-30T12:45:20.470964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:45:42.325441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377353+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:20.377353+0000","last_peered":"2023-10-30T12:45:20.377353+0000","last_clean":"2023-10-30T12:45:20.377353+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:20.377353+0000","last_undegraded":"2023-10-30T12:45:20.377353+0000","last_fullsized":"2023-10-30T12:45:20.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:59.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069882+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:20.069882+0000","last_peered":"2023-10-30T12:45:20.069882+0000","last_clean":"2023-10-30T12:45:20.069882+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:20.069882+0000","last_undegraded":"2023-10-30T12:45:20.069882+0000","last_fullsized":"2023-10-30T12:45:20.069882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:09.263782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376997+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:20.376997+0000","last_peered":"2023-10-30T12:45:20.376997+0000","last_clean":"2023-10-30T12:45:20.376997+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:20.376997+0000","last_undegraded":"2023-10-30T12:45:20.376997+0000","last_fullsized":"2023-10-30T12:45:20.376997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:48:04.677113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069348+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:20.069348+0000","last_peered":"2023-10-30T12:45:20.069348+0000","last_clean":"2023-10-30T12:45:20.069348+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:20.069348+0000","last_undegraded":"2023-10-30T12:45:20.069348+0000","last_fullsized":"2023-10-30T12:45:20.069348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:43:52.826848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470720+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:20.470720+0000","last_peered":"2023-10-30T12:45:20.470720+0000","last_clean":"2023-10-30T12:45:20.470720+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:20.470720+0000","last_undegraded":"2023-10-30T12:45:20.470720+0000","last_fullsized":"2023-10-30T12:45:20.470720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:27:17.979749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072297+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:20.072297+0000","last_peered":"2023-10-30T12:45:20.072297+0000","last_clean":"2023-10-30T12:45:20.072297+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:20.072297+0000","last_undegraded":"2023-10-30T12:45:20.072297+0000","last_fullsized":"2023-10-30T12:45:20.072297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:28.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471673+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:20.471673+0000","last_peered":"2023-10-30T12:45:20.471673+0000","last_clean":"2023-10-30T12:45:20.471673+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:20.471673+0000","last_undegraded":"2023-10-30T12:45:20.471673+0000","last_fullsized":"2023-10-30T12:45:20.471673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:06:39.736671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471336+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:20.471336+0000","last_peered":"2023-10-30T12:45:20.471336+0000","last_clean":"2023-10-30T12:45:20.471336+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:20.471336+0000","last_undegraded":"2023-10-30T12:45:20.471336+0000","last_fullsized":"2023-10-30T12:45:20.471336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:55.158597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071808+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:20.071808+0000","last_peered":"2023-10-30T12:45:20.071808+0000","last_clean":"2023-10-30T12:45:20.071808+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:20.071808+0000","last_undegraded":"2023-10-30T12:45:20.071808+0000","last_fullsized":"2023-10-30T12:45:20.071808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:18:07.085147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470864+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:20.470864+0000","last_peered":"2023-10-30T12:45:20.470864+0000","last_clean":"2023-10-30T12:45:20.470864+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:20.470864+0000","last_undegraded":"2023-10-30T12:45:20.470864+0000","last_fullsized":"2023-10-30T12:45:20.470864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:45:04.008631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470638+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:20.470638+0000","last_peered":"2023-10-30T12:45:20.470638+0000","last_clean":"2023-10-30T12:45:20.470638+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:20.470638+0000","last_undegraded":"2023-10-30T12:45:20.470638+0000","last_fullsized":"2023-10-30T12:45:20.470638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:02:04.825098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:23.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376825+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:20.376825+0000","last_peered":"2023-10-30T12:45:20.376825+0000","last_clean":"2023-10-30T12:45:20.376825+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:20.376825+0000","last_undegraded":"2023-10-30T12:45:20.376825+0000","last_fullsized":"2023-10-30T12:45:20.376825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:48:16.344709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377969+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:20.377969+0000","last_peered":"2023-10-30T12:45:20.377969+0000","last_clean":"2023-10-30T12:45:20.377969+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:20.377969+0000","last_undegraded":"2023-10-30T12:45:20.377969+0000","last_fullsized":"2023-10-30T12:45:20.377969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:08.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471127+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:20.471127+0000","last_peered":"2023-10-30T12:45:20.471127+0000","last_clean":"2023-10-30T12:45:20.471127+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:20.471127+0000","last_undegraded":"2023-10-30T12:45:20.471127+0000","last_fullsized":"2023-10-30T12:45:20.471127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:05.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070768+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:20.070768+0000","last_peered":"2023-10-30T12:45:20.070768+0000","last_clean":"2023-10-30T12:45:20.070768+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:20.070768+0000","last_undegraded":"2023-10-30T12:45:20.070768+0000","last_fullsized":"2023-10-30T12:45:20.070768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:17.483158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471411+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:20.471411+0000","last_peered":"2023-10-30T12:45:20.471411+0000","last_clean":"2023-10-30T12:45:20.471411+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:20.471411+0000","last_undegraded":"2023-10-30T12:45:20.471411+0000","last_fullsized":"2023-10-30T12:45:20.471411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:16:21.392983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470962+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:20.470962+0000","last_peered":"2023-10-30T12:45:20.470962+0000","last_clean":"2023-10-30T12:45:20.470962+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:20.470962+0000","last_undegraded":"2023-10-30T12:45:20.470962+0000","last_fullsized":"2023-10-30T12:45:20.470962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:02.564276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377142+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:20.377142+0000","last_peered":"2023-10-30T12:45:20.377142+0000","last_clean":"2023-10-30T12:45:20.377142+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:20.377142+0000","last_undegraded":"2023-10-30T12:45:20.377142+0000","last_fullsized":"2023-10-30T12:45:20.377142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:33:03.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470206+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:20.470206+0000","last_peered":"2023-10-30T12:45:20.470206+0000","last_clean":"2023-10-30T12:45:20.470206+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:20.470206+0000","last_undegraded":"2023-10-30T12:45:20.470206+0000","last_fullsized":"2023-10-30T12:45:20.470206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:31:49.660950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376996+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:20.376996+0000","last_peered":"2023-10-30T12:45:20.376996+0000","last_clean":"2023-10-30T12:45:20.376996+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:20.376996+0000","last_undegraded":"2023-10-30T12:45:20.376996+0000","last_fullsized":"2023-10-30T12:45:20.376996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:51.286544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471712+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:20.471712+0000","last_peered":"2023-10-30T12:45:20.471712+0000","last_clean":"2023-10-30T12:45:20.471712+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:20.471712+0000","last_undegraded":"2023-10-30T12:45:20.471712+0000","last_fullsized":"2023-10-30T12:45:20.471712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:57.636148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471671+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:20.471671+0000","last_peered":"2023-10-30T12:45:20.471671+0000","last_clean":"2023-10-30T12:45:20.471671+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:20.471671+0000","last_undegraded":"2023-10-30T12:45:20.471671+0000","last_fullsized":"2023-10-30T12:45:20.471671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:30.410457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067039+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:20.067039+0000","last_peered":"2023-10-30T12:45:20.067039+0000","last_clean":"2023-10-30T12:45:20.067039+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:20.067039+0000","last_undegraded":"2023-10-30T12:45:20.067039+0000","last_fullsized":"2023-10-30T12:45:20.067039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:46.788124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471494+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:20.471494+0000","last_peered":"2023-10-30T12:45:20.471494+0000","last_clean":"2023-10-30T12:45:20.471494+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:20.471494+0000","last_undegraded":"2023-10-30T12:45:20.471494+0000","last_fullsized":"2023-10-30T12:45:20.471494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:44.555138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471084+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:20.471084+0000","last_peered":"2023-10-30T12:45:20.471084+0000","last_clean":"2023-10-30T12:45:20.471084+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:20.471084+0000","last_undegraded":"2023-10-30T12:45:20.471084+0000","last_fullsized":"2023-10-30T12:45:20.471084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:29.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377430+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:20.377430+0000","last_peered":"2023-10-30T12:45:20.377430+0000","last_clean":"2023-10-30T12:45:20.377430+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:20.377430+0000","last_undegraded":"2023-10-30T12:45:20.377430+0000","last_fullsized":"2023-10-30T12:45:20.377430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:50:53.563609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471353+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:20.471353+0000","last_peered":"2023-10-30T12:45:20.471353+0000","last_clean":"2023-10-30T12:45:20.471353+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:20.471353+0000","last_undegraded":"2023-10-30T12:45:20.471353+0000","last_fullsized":"2023-10-30T12:45:20.471353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:12:59.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470912+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:20.470912+0000","last_peered":"2023-10-30T12:45:20.470912+0000","last_clean":"2023-10-30T12:45:20.470912+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:20.470912+0000","last_undegraded":"2023-10-30T12:45:20.470912+0000","last_fullsized":"2023-10-30T12:45:20.470912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:10.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071561+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:20.071561+0000","last_peered":"2023-10-30T12:45:20.071561+0000","last_clean":"2023-10-30T12:45:20.071561+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:20.071561+0000","last_undegraded":"2023-10-30T12:45:20.071561+0000","last_fullsized":"2023-10-30T12:45:20.071561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:17:56.183441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471094+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:20.471094+0000","last_peered":"2023-10-30T12:45:20.471094+0000","last_clean":"2023-10-30T12:45:20.471094+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:20.471094+0000","last_undegraded":"2023-10-30T12:45:20.471094+0000","last_fullsized":"2023-10-30T12:45:20.471094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471141+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:20.471141+0000","last_peered":"2023-10-30T12:45:20.471141+0000","last_clean":"2023-10-30T12:45:20.471141+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:20.471141+0000","last_undegraded":"2023-10-30T12:45:20.471141+0000","last_fullsized":"2023-10-30T12:45:20.471141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:23:00.831740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630920+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:19.630920+0000","last_peered":"2023-10-30T12:45:19.630920+0000","last_clean":"2023-10-30T12:45:19.630920+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:19.630920+0000","last_undegraded":"2023-10-30T12:45:19.630920+0000","last_fullsized":"2023-10-30T12:45:19.630920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470208+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:20.470208+0000","last_peered":"2023-10-30T12:45:20.470208+0000","last_clean":"2023-10-30T12:45:20.470208+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:20.470208+0000","last_undegraded":"2023-10-30T12:45:20.470208+0000","last_fullsized":"2023-10-30T12:45:20.470208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:26.174484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633996+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:19.633996+0000","last_peered":"2023-10-30T12:45:19.633996+0000","last_clean":"2023-10-30T12:45:19.633996+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:19.633996+0000","last_undegraded":"2023-10-30T12:45:19.633996+0000","last_fullsized":"2023-10-30T12:45:19.633996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:12:12.444137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633919+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:19.633919+0000","last_peered":"2023-10-30T12:45:19.633919+0000","last_clean":"2023-10-30T12:45:19.633919+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:19.633919+0000","last_undegraded":"2023-10-30T12:45:19.633919+0000","last_fullsized":"2023-10-30T12:45:19.633919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377703+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:20.377703+0000","last_peered":"2023-10-30T12:45:20.377703+0000","last_clean":"2023-10-30T12:45:20.377703+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:20.377703+0000","last_undegraded":"2023-10-30T12:45:20.377703+0000","last_fullsized":"2023-10-30T12:45:20.377703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:40:41.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634489+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:19.634489+0000","last_peered":"2023-10-30T12:45:19.634489+0000","last_clean":"2023-10-30T12:45:19.634489+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:19.634489+0000","last_undegraded":"2023-10-30T12:45:19.634489+0000","last_fullsized":"2023-10-30T12:45:19.634489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471307+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:20.471307+0000","last_peered":"2023-10-30T12:45:20.471307+0000","last_clean":"2023-10-30T12:45:20.471307+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:20.471307+0000","last_undegraded":"2023-10-30T12:45:20.471307+0000","last_fullsized":"2023-10-30T12:45:20.471307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:07:09.393386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.081621+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:24.081621+0000","last_peered":"2023-10-30T12:45:24.081621+0000","last_clean":"2023-10-30T12:45:24.081621+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:24.081621+0000","last_undegraded":"2023-10-30T12:45:24.081621+0000","last_fullsized":"2023-10-30T12:45:24.081621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072148+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:20.072148+0000","last_peered":"2023-10-30T12:45:20.072148+0000","last_clean":"2023-10-30T12:45:20.072148+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:20.072148+0000","last_undegraded":"2023-10-30T12:45:20.072148+0000","last_fullsized":"2023-10-30T12:45:20.072148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:17.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.085227+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:24.085227+0000","last_peered":"2023-10-30T12:45:24.085227+0000","last_clean":"2023-10-30T12:45:24.085227+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:24.085227+0000","last_undegraded":"2023-10-30T12:45:24.085227+0000","last_fullsized":"2023-10-30T12:45:24.085227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069876+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:20.069876+0000","last_peered":"2023-10-30T12:45:20.069876+0000","last_clean":"2023-10-30T12:45:20.069876+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:20.069876+0000","last_undegraded":"2023-10-30T12:45:20.069876+0000","last_fullsized":"2023-10-30T12:45:20.069876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:07:42.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.083457+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:24.083457+0000","last_peered":"2023-10-30T12:45:24.083457+0000","last_clean":"2023-10-30T12:45:24.083457+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:24.083457+0000","last_undegraded":"2023-10-30T12:45:24.083457+0000","last_fullsized":"2023-10-30T12:45:24.083457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470757+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:20.470757+0000","last_peered":"2023-10-30T12:45:20.470757+0000","last_clean":"2023-10-30T12:45:20.470757+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:20.470757+0000","last_undegraded":"2023-10-30T12:45:20.470757+0000","last_fullsized":"2023-10-30T12:45:20.470757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:17.831426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634163+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:19.634163+0000","last_peered":"2023-10-30T12:45:19.634163+0000","last_clean":"2023-10-30T12:45:19.634163+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:19.634163+0000","last_undegraded":"2023-10-30T12:45:19.634163+0000","last_fullsized":"2023-10-30T12:45:19.634163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070431+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:20.070431+0000","last_peered":"2023-10-30T12:45:20.070431+0000","last_clean":"2023-10-30T12:45:20.070431+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:20.070431+0000","last_undegraded":"2023-10-30T12:45:20.070431+0000","last_fullsized":"2023-10-30T12:45:20.070431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:14:31.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633547+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:19.633547+0000","last_peered":"2023-10-30T12:45:19.633547+0000","last_clean":"2023-10-30T12:45:19.633547+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:19.633547+0000","last_undegraded":"2023-10-30T12:45:19.633547+0000","last_fullsized":"2023-10-30T12:45:19.633547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470957+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:20.470957+0000","last_peered":"2023-10-30T12:45:20.470957+0000","last_clean":"2023-10-30T12:45:20.470957+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:20.470957+0000","last_undegraded":"2023-10-30T12:45:20.470957+0000","last_fullsized":"2023-10-30T12:45:20.470957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:13.282854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.080427+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:24.080427+0000","last_peered":"2023-10-30T12:45:24.080427+0000","last_clean":"2023-10-30T12:45:24.080427+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:24.080427+0000","last_undegraded":"2023-10-30T12:45:24.080427+0000","last_fullsized":"2023-10-30T12:45:24.080427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069448+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:20.069448+0000","last_peered":"2023-10-30T12:45:20.069448+0000","last_clean":"2023-10-30T12:45:20.069448+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:20.069448+0000","last_undegraded":"2023-10-30T12:45:20.069448+0000","last_fullsized":"2023-10-30T12:45:20.069448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:14.563685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.073220+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:20.073220+0000","last_peered":"2023-10-30T12:45:20.073220+0000","last_clean":"2023-10-30T12:45:20.073220+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:20.073220+0000","last_undegraded":"2023-10-30T12:45:20.073220+0000","last_fullsized":"2023-10-30T12:45:20.073220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071276+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:20.071276+0000","last_peered":"2023-10-30T12:45:20.071276+0000","last_clean":"2023-10-30T12:45:20.071276+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:20.071276+0000","last_undegraded":"2023-10-30T12:45:20.071276+0000","last_fullsized":"2023-10-30T12:45:20.071276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:43.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.077908+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:24.077908+0000","last_peered":"2023-10-30T12:45:24.077908+0000","last_clean":"2023-10-30T12:45:24.077908+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:24.077908+0000","last_undegraded":"2023-10-30T12:45:24.077908+0000","last_fullsized":"2023-10-30T12:45:24.077908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072543+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:20.072543+0000","last_peered":"2023-10-30T12:45:20.072543+0000","last_clean":"2023-10-30T12:45:20.072543+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:20.072543+0000","last_undegraded":"2023-10-30T12:45:20.072543+0000","last_fullsized":"2023-10-30T12:45:20.072543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:54.641020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067271+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:20.067271+0000","last_peered":"2023-10-30T12:45:20.067271+0000","last_clean":"2023-10-30T12:45:20.067271+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:20.067271+0000","last_undegraded":"2023-10-30T12:45:20.067271+0000","last_fullsized":"2023-10-30T12:45:20.067271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378262+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:20.378262+0000","last_peered":"2023-10-30T12:45:20.378262+0000","last_clean":"2023-10-30T12:45:20.378262+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:20.378262+0000","last_undegraded":"2023-10-30T12:45:20.378262+0000","last_fullsized":"2023-10-30T12:45:20.378262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:08.395587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630441+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:19.630441+0000","last_peered":"2023-10-30T12:45:19.630441+0000","last_clean":"2023-10-30T12:45:19.630441+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:19.630441+0000","last_undegraded":"2023-10-30T12:45:19.630441+0000","last_fullsized":"2023-10-30T12:45:19.630441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069897+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:20.069897+0000","last_peered":"2023-10-30T12:45:20.069897+0000","last_clean":"2023-10-30T12:45:20.069897+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:20.069897+0000","last_undegraded":"2023-10-30T12:45:20.069897+0000","last_fullsized":"2023-10-30T12:45:20.069897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:58:00.569432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071157+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:19.071157+0000","last_peered":"2023-10-30T12:45:19.071157+0000","last_clean":"2023-10-30T12:45:19.071157+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:19.071157+0000","last_undegraded":"2023-10-30T12:45:19.071157+0000","last_fullsized":"2023-10-30T12:45:19.071157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634249+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:19.634249+0000","last_peered":"2023-10-30T12:45:19.634249+0000","last_clean":"2023-10-30T12:45:19.634249+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:19.634249+0000","last_undegraded":"2023-10-30T12:45:19.634249+0000","last_fullsized":"2023-10-30T12:45:19.634249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470630+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:20.470630+0000","last_peered":"2023-10-30T12:45:20.470630+0000","last_clean":"2023-10-30T12:45:20.470630+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:20.470630+0000","last_undegraded":"2023-10-30T12:45:20.470630+0000","last_fullsized":"2023-10-30T12:45:20.470630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:25:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634164+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:19.634164+0000","last_peered":"2023-10-30T12:45:19.634164+0000","last_clean":"2023-10-30T12:45:19.634164+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:19.634164+0000","last_undegraded":"2023-10-30T12:45:19.634164+0000","last_fullsized":"2023-10-30T12:45:19.634164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470604+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:20.470604+0000","last_peered":"2023-10-30T12:45:20.470604+0000","last_clean":"2023-10-30T12:45:20.470604+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:20.470604+0000","last_undegraded":"2023-10-30T12:45:20.470604+0000","last_fullsized":"2023-10-30T12:45:20.470604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:55.822788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630426+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:19.630426+0000","last_peered":"2023-10-30T12:45:19.630426+0000","last_clean":"2023-10-30T12:45:19.630426+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:19.630426+0000","last_undegraded":"2023-10-30T12:45:19.630426+0000","last_fullsized":"2023-10-30T12:45:19.630426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066971+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:20.066971+0000","last_peered":"2023-10-30T12:45:20.066971+0000","last_clean":"2023-10-30T12:45:20.066971+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:20.066971+0000","last_undegraded":"2023-10-30T12:45:20.066971+0000","last_fullsized":"2023-10-30T12:45:20.066971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:17:16.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634197+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:19.634197+0000","last_peered":"2023-10-30T12:45:19.634197+0000","last_clean":"2023-10-30T12:45:19.634197+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:19.634197+0000","last_undegraded":"2023-10-30T12:45:19.634197+0000","last_fullsized":"2023-10-30T12:45:19.634197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072149+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:20.072149+0000","last_peered":"2023-10-30T12:45:20.072149+0000","last_clean":"2023-10-30T12:45:20.072149+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:20.072149+0000","last_undegraded":"2023-10-30T12:45:20.072149+0000","last_fullsized":"2023-10-30T12:45:20.072149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630601+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.630601+0000","last_peered":"2023-10-30T12:45:19.630601+0000","last_clean":"2023-10-30T12:45:19.630601+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.630601+0000","last_undegraded":"2023-10-30T12:45:19.630601+0000","last_fullsized":"2023-10-30T12:45:19.630601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378093+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:20.378093+0000","last_peered":"2023-10-30T12:45:20.378093+0000","last_clean":"2023-10-30T12:45:20.378093+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:20.378093+0000","last_undegraded":"2023-10-30T12:45:20.378093+0000","last_fullsized":"2023-10-30T12:45:20.378093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:05.357577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634481+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:19.634481+0000","last_peered":"2023-10-30T12:45:19.634481+0000","last_clean":"2023-10-30T12:45:19.634481+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:19.634481+0000","last_undegraded":"2023-10-30T12:45:19.634481+0000","last_fullsized":"2023-10-30T12:45:19.634481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066974+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:20.066974+0000","last_peered":"2023-10-30T12:45:20.066974+0000","last_clean":"2023-10-30T12:45:20.066974+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:20.066974+0000","last_undegraded":"2023-10-30T12:45:20.066974+0000","last_fullsized":"2023-10-30T12:45:20.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:46:43.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630684+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:19.630684+0000","last_peered":"2023-10-30T12:45:19.630684+0000","last_clean":"2023-10-30T12:45:19.630684+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:19.630684+0000","last_undegraded":"2023-10-30T12:45:19.630684+0000","last_fullsized":"2023-10-30T12:45:19.630684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471700+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:20.471700+0000","last_peered":"2023-10-30T12:45:20.471700+0000","last_clean":"2023-10-30T12:45:20.471700+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:20.471700+0000","last_undegraded":"2023-10-30T12:45:20.471700+0000","last_fullsized":"2023-10-30T12:45:20.471700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:25.320280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.082644+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:24.082644+0000","last_peered":"2023-10-30T12:45:24.082644+0000","last_clean":"2023-10-30T12:45:24.082644+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:24.082644+0000","last_undegraded":"2023-10-30T12:45:24.082644+0000","last_fullsized":"2023-10-30T12:45:24.082644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471488+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:20.471488+0000","last_peered":"2023-10-30T12:45:20.471488+0000","last_clean":"2023-10-30T12:45:20.471488+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:20.471488+0000","last_undegraded":"2023-10-30T12:45:20.471488+0000","last_fullsized":"2023-10-30T12:45:20.471488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:14.407746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634110+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:19.634110+0000","last_peered":"2023-10-30T12:45:19.634110+0000","last_clean":"2023-10-30T12:45:19.634110+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:19.634110+0000","last_undegraded":"2023-10-30T12:45:19.634110+0000","last_fullsized":"2023-10-30T12:45:19.634110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071773+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:20.071773+0000","last_peered":"2023-10-30T12:45:20.071773+0000","last_clean":"2023-10-30T12:45:20.071773+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:20.071773+0000","last_undegraded":"2023-10-30T12:45:20.071773+0000","last_fullsized":"2023-10-30T12:45:20.071773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:09.037055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630574+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:19.630574+0000","last_peered":"2023-10-30T12:45:19.630574+0000","last_clean":"2023-10-30T12:45:19.630574+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:19.630574+0000","last_undegraded":"2023-10-30T12:45:19.630574+0000","last_fullsized":"2023-10-30T12:45:19.630574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470363+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:20.470363+0000","last_peered":"2023-10-30T12:45:20.470363+0000","last_clean":"2023-10-30T12:45:20.470363+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:20.470363+0000","last_undegraded":"2023-10-30T12:45:20.470363+0000","last_fullsized":"2023-10-30T12:45:20.470363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:45:36.069824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630584+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:19.630584+0000","last_peered":"2023-10-30T12:45:19.630584+0000","last_clean":"2023-10-30T12:45:19.630584+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:19.630584+0000","last_undegraded":"2023-10-30T12:45:19.630584+0000","last_fullsized":"2023-10-30T12:45:19.630584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:21.343385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084421+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:24.084421+0000","last_peered":"2023-10-30T12:45:24.084421+0000","last_clean":"2023-10-30T12:45:24.084421+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:24.084421+0000","last_undegraded":"2023-10-30T12:45:24.084421+0000","last_fullsized":"2023-10-30T12:45:24.084421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470979+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:20.470979+0000","last_peered":"2023-10-30T12:45:20.470979+0000","last_clean":"2023-10-30T12:45:20.470979+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:20.470979+0000","last_undegraded":"2023-10-30T12:45:20.470979+0000","last_fullsized":"2023-10-30T12:45:20.470979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:44.769458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377742+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:20.377742+0000","last_peered":"2023-10-30T12:45:20.377742+0000","last_clean":"2023-10-30T12:45:20.377742+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:20.377742+0000","last_undegraded":"2023-10-30T12:45:20.377742+0000","last_fullsized":"2023-10-30T12:45:20.377742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:13.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084274+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:24.084274+0000","last_peered":"2023-10-30T12:45:24.084274+0000","last_clean":"2023-10-30T12:45:24.084274+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:24.084274+0000","last_undegraded":"2023-10-30T12:45:24.084274+0000","last_fullsized":"2023-10-30T12:45:24.084274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377958+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:20.377958+0000","last_peered":"2023-10-30T12:45:20.377958+0000","last_clean":"2023-10-30T12:45:20.377958+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:20.377958+0000","last_undegraded":"2023-10-30T12:45:20.377958+0000","last_fullsized":"2023-10-30T12:45:20.377958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:08.137679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.085394+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:24.085394+0000","last_peered":"2023-10-30T12:45:24.085394+0000","last_clean":"2023-10-30T12:45:24.085394+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:24.085394+0000","last_undegraded":"2023-10-30T12:45:24.085394+0000","last_fullsized":"2023-10-30T12:45:24.085394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471292+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:20.471292+0000","last_peered":"2023-10-30T12:45:20.471292+0000","last_clean":"2023-10-30T12:45:20.471292+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:20.471292+0000","last_undegraded":"2023-10-30T12:45:20.471292+0000","last_fullsized":"2023-10-30T12:45:20.471292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:26.117300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471675+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:20.471675+0000","last_peered":"2023-10-30T12:45:20.471675+0000","last_clean":"2023-10-30T12:45:20.471675+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:20.471675+0000","last_undegraded":"2023-10-30T12:45:20.471675+0000","last_fullsized":"2023-10-30T12:45:20.471675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471526+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:20.471526+0000","last_peered":"2023-10-30T12:45:20.471526+0000","last_clean":"2023-10-30T12:45:20.471526+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:20.471526+0000","last_undegraded":"2023-10-30T12:45:20.471526+0000","last_fullsized":"2023-10-30T12:45:20.471526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:57:18.999410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.082716+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:24.082716+0000","last_peered":"2023-10-30T12:45:24.082716+0000","last_clean":"2023-10-30T12:45:24.082716+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:24.082716+0000","last_undegraded":"2023-10-30T12:45:24.082716+0000","last_fullsized":"2023-10-30T12:45:24.082716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7919999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.964}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:30.003 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:30.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.177+0000 7fcf14417640 1 Processor -- start 2023-10-30T12:45:30.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.177+0000 7fcf14417640 1 -- start start 2023-10-30T12:45:30.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf14417640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf0c153c80 0x7fcf0c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf14417640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c159f20 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf14417640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c15d620 con 0x7fcf0c0c42a0 2023-10-30T12:45:30.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf14417640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c15dec0 con 0x7fcf0c153c80 2023-10-30T12:45:30.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf119b2640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fcf0c0c4a10 0x7fcf0c15a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53888/0 (socket says 172.21.15.8:53888) 2023-10-30T12:45:30.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf119b2640 1 -- 172.21.15.8:0/773171702 learned_addr learned my addr 172.21.15.8:0/773171702 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:30.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 --2- 172.21.15.8:0/773171702 >> v2:172.21.15.46:3300/0 conn(0x7fcf0c153c80 0x7fcf0c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540489080 0 0) 0x7fcf0c159f20 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcef8003530 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2765133664 0 0) 0x7fcef8003530 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 >> v2:172.21.15.46:3300/0 conn(0x7fcf0c153c80 msgr2=0x7fcf0c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 --2- 172.21.15.8:0/773171702 >> v2:172.21.15.46:3300/0 conn(0x7fcf0c153c80 0x7fcf0c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 >> v1:172.21.15.46:6789/0 conn(0x7fcf0c0c42a0 legacy=0x7fcf0c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c15d900 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fcefc001680 con 0x7fcf0c0c4a10 2023-10-30T12:45:30.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 >> v1:172.21.15.8:6789/0 conn(0x7fcf0c0c4a10 legacy=0x7fcf0c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef80047d0 0x7fcef8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcef8005190 0x7fcef800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c15dec0 con 0x7fcef8003980 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef800c040 con 0x7fcef80047d0 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef800c200 con 0x7fcef8005190 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf129b4640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcef8005190 0x7fcef800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 -- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcef8005190 msgr2=0x7fcef800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcef8005190 0x7fcef800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 -- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef80047d0 msgr2=0x7fcef8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef80047d0 0x7fcef8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf121b3640 1 -- 172.21.15.8:0/773171702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8003530 con 0x7fcef8003980 2023-10-30T12:45:30.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.181+0000 7fcf129b4640 1 -- 172.21.15.8:0/773171702 reap_dead start 2023-10-30T12:45:30.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf121b3640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009ae0 tx=0x7fcf0801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb14ccb398c6c9a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf080a5640 con 0x7fcef8003980 2023-10-30T12:45:30.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf080a57a0 con 0x7fcef8003980 2023-10-30T12:45:30.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf111b1640 1 -- 172.21.15.8:0/773171702 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf080a5b50 con 0x7fcef8003980 2023-10-30T12:45:30.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 -- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 msgr2=0x7fcef8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009ae0 tx=0x7fcf0801ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:30.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 -- 172.21.15.8:0/773171702 shutdown_connections 2023-10-30T12:45:30.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 --2- 172.21.15.8:0/773171702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 -- 172.21.15.8:0/773171702 >> 172.21.15.8:0/773171702 conn(0x7fcf0c0715c0 msgr2=0x7fcf0c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:30.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 -- 172.21.15.8:0/773171702 shutdown_connections 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.185+0000 7fcf14417640 1 -- 172.21.15.8:0/773171702 wait complete. 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 Processor -- start 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- start start 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef8003980 0x7fcf0c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcf0c15b7e0 0x7fcf0c148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c156ed0 con 0x7fcf0c146c30 2023-10-30T12:45:30.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c15afa0 con 0x7fcef8003980 2023-10-30T12:45:30.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c1562c0 con 0x7fcf0c15b7e0 2023-10-30T12:45:30.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55930/0 (socket says 172.21.15.8:55930) 2023-10-30T12:45:30.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 -- 172.21.15.8:0/3152850206 learned_addr learned my addr 172.21.15.8:0/3152850206 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:30.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf121b3640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef8003980 0x7fcf0c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf119b2640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcf0c15b7e0 0x7fcf0c148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 -- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcf0c15b7e0 msgr2=0x7fcf0c148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcf0c15b7e0 0x7fcf0c148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 -- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef8003980 msgr2=0x7fcf0c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef8003980 0x7fcf0c1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 -- 172.21.15.8:0/3152850206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c1565a0 con 0x7fcf0c146c30 2023-10-30T12:45:30.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf129b4640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c08d1f0 tx=0x7fcf0400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003b90 con 0x7fcf0c146c30 2023-10-30T12:45:30.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf04003d10 con 0x7fcf0c146c30 2023-10-30T12:45:30.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c156880 con 0x7fcf0c146c30 2023-10-30T12:45:30.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04005310 con 0x7fcf0c146c30 2023-10-30T12:45:30.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c1491e0 con 0x7fcf0c146c30 2023-10-30T12:45:30.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.189+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c1f22a0 con 0x7fcf0c146c30 2023-10-30T12:45:30.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.193+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf04005470 con 0x7fcf0c146c30 2023-10-30T12:45:30.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.193+0000 7fcf02ffd640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.193+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf04083b10 con 0x7fcf0c146c30 2023-10-30T12:45:30.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.193+0000 7fcf121b3640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.193+0000 7fcf121b3640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 0x7fcef0068960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf080af000 tx=0x7fcf080046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.197+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf04050f10 con 0x7fcf0c146c30 2023-10-30T12:45:30.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.349+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf0c156d30 con 0x7fcef00664a0 2023-10-30T12:45:30.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.357+0000 7fcf02ffd640 1 -- 172.21.15.8:0/3152850206 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264535 (secure 0 0 0) 0x7fcf080a5910 con 0x7fcef00664a0 2023-10-30T12:45:30.362 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:30.364 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:30.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 msgr2=0x7fcef0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 0x7fcef0068960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf080af000 tx=0x7fcf080046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 msgr2=0x7fcf0c1470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c08d1f0 tx=0x7fcf0400e9e0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 shutdown_connections 2023-10-30T12:45:30.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcef00664a0 0x7fcef0068960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf0c146c30 0x7fcf0c1470b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcf0c15b7e0 0x7fcf0c148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 --2- 172.21.15.8:0/3152850206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef8003980 0x7fcf0c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 >> 172.21.15.8:0/3152850206 conn(0x7fcf0c0715c0 msgr2=0x7fcf0c1526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:30.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 shutdown_connections 2023-10-30T12:45:30.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.361+0000 7fcf14417640 1 -- 172.21.15.8:0/3152850206 wait complete. 2023-10-30T12:45:30.382 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-30T12:45:29.614691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252228,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722492,"statfs":{"total":773094113280,"available":772835831808,"internally_reserved":0,"allocated":2887680,"data_stored":1751721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005823"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071955+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:20.071955+0000","last_peered":"2023-10-30T12:45:20.071955+0000","last_clean":"2023-10-30T12:45:20.071955+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:20.071955+0000","last_undegraded":"2023-10-30T12:45:20.071955+0000","last_fullsized":"2023-10-30T12:45:20.071955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:33:48.910689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067349+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:20.067349+0000","last_peered":"2023-10-30T12:45:20.067349+0000","last_clean":"2023-10-30T12:45:20.067349+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:20.067349+0000","last_undegraded":"2023-10-30T12:45:20.067349+0000","last_fullsized":"2023-10-30T12:45:20.067349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:34.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470964+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:20.470964+0000","last_peered":"2023-10-30T12:45:20.470964+0000","last_clean":"2023-10-30T12:45:20.470964+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:20.470964+0000","last_undegraded":"2023-10-30T12:45:20.470964+0000","last_fullsized":"2023-10-30T12:45:20.470964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:45:42.325441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377353+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:20.377353+0000","last_peered":"2023-10-30T12:45:20.377353+0000","last_clean":"2023-10-30T12:45:20.377353+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:20.377353+0000","last_undegraded":"2023-10-30T12:45:20.377353+0000","last_fullsized":"2023-10-30T12:45:20.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:59.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069882+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:20.069882+0000","last_peered":"2023-10-30T12:45:20.069882+0000","last_clean":"2023-10-30T12:45:20.069882+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:20.069882+0000","last_undegraded":"2023-10-30T12:45:20.069882+0000","last_fullsized":"2023-10-30T12:45:20.069882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:09.263782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376997+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:20.376997+0000","last_peered":"2023-10-30T12:45:20.376997+0000","last_clean":"2023-10-30T12:45:20.376997+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:20.376997+0000","last_undegraded":"2023-10-30T12:45:20.376997+0000","last_fullsized":"2023-10-30T12:45:20.376997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:48:04.677113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069348+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:20.069348+0000","last_peered":"2023-10-30T12:45:20.069348+0000","last_clean":"2023-10-30T12:45:20.069348+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:20.069348+0000","last_undegraded":"2023-10-30T12:45:20.069348+0000","last_fullsized":"2023-10-30T12:45:20.069348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:43:52.826848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470720+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:20.470720+0000","last_peered":"2023-10-30T12:45:20.470720+0000","last_clean":"2023-10-30T12:45:20.470720+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:20.470720+0000","last_undegraded":"2023-10-30T12:45:20.470720+0000","last_fullsized":"2023-10-30T12:45:20.470720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:27:17.979749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072297+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:20.072297+0000","last_peered":"2023-10-30T12:45:20.072297+0000","last_clean":"2023-10-30T12:45:20.072297+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:20.072297+0000","last_undegraded":"2023-10-30T12:45:20.072297+0000","last_fullsized":"2023-10-30T12:45:20.072297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:28.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471673+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:20.471673+0000","last_peered":"2023-10-30T12:45:20.471673+0000","last_clean":"2023-10-30T12:45:20.471673+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:20.471673+0000","last_undegraded":"2023-10-30T12:45:20.471673+0000","last_fullsized":"2023-10-30T12:45:20.471673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:06:39.736671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471336+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:20.471336+0000","last_peered":"2023-10-30T12:45:20.471336+0000","last_clean":"2023-10-30T12:45:20.471336+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:20.471336+0000","last_undegraded":"2023-10-30T12:45:20.471336+0000","last_fullsized":"2023-10-30T12:45:20.471336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:55.158597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071808+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:20.071808+0000","last_peered":"2023-10-30T12:45:20.071808+0000","last_clean":"2023-10-30T12:45:20.071808+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:20.071808+0000","last_undegraded":"2023-10-30T12:45:20.071808+0000","last_fullsized":"2023-10-30T12:45:20.071808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:18:07.085147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470864+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:20.470864+0000","last_peered":"2023-10-30T12:45:20.470864+0000","last_clean":"2023-10-30T12:45:20.470864+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:20.470864+0000","last_undegraded":"2023-10-30T12:45:20.470864+0000","last_fullsized":"2023-10-30T12:45:20.470864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:45:04.008631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470638+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:20.470638+0000","last_peered":"2023-10-30T12:45:20.470638+0000","last_clean":"2023-10-30T12:45:20.470638+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:20.470638+0000","last_undegraded":"2023-10-30T12:45:20.470638+0000","last_fullsized":"2023-10-30T12:45:20.470638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:02:04.825098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:23.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376825+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:20.376825+0000","last_peered":"2023-10-30T12:45:20.376825+0000","last_clean":"2023-10-30T12:45:20.376825+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:20.376825+0000","last_undegraded":"2023-10-30T12:45:20.376825+0000","last_fullsized":"2023-10-30T12:45:20.376825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:48:16.344709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377969+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:20.377969+0000","last_peered":"2023-10-30T12:45:20.377969+0000","last_clean":"2023-10-30T12:45:20.377969+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:20.377969+0000","last_undegraded":"2023-10-30T12:45:20.377969+0000","last_fullsized":"2023-10-30T12:45:20.377969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:08.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471127+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:20.471127+0000","last_peered":"2023-10-30T12:45:20.471127+0000","last_clean":"2023-10-30T12:45:20.471127+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:20.471127+0000","last_undegraded":"2023-10-30T12:45:20.471127+0000","last_fullsized":"2023-10-30T12:45:20.471127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:05.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070768+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:20.070768+0000","last_peered":"2023-10-30T12:45:20.070768+0000","last_clean":"2023-10-30T12:45:20.070768+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:20.070768+0000","last_undegraded":"2023-10-30T12:45:20.070768+0000","last_fullsized":"2023-10-30T12:45:20.070768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:17.483158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471411+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:20.471411+0000","last_peered":"2023-10-30T12:45:20.471411+0000","last_clean":"2023-10-30T12:45:20.471411+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:20.471411+0000","last_undegraded":"2023-10-30T12:45:20.471411+0000","last_fullsized":"2023-10-30T12:45:20.471411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:16:21.392983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470962+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:20.470962+0000","last_peered":"2023-10-30T12:45:20.470962+0000","last_clean":"2023-10-30T12:45:20.470962+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:20.470962+0000","last_undegraded":"2023-10-30T12:45:20.470962+0000","last_fullsized":"2023-10-30T12:45:20.470962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:02.564276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377142+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:20.377142+0000","last_peered":"2023-10-30T12:45:20.377142+0000","last_clean":"2023-10-30T12:45:20.377142+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:20.377142+0000","last_undegraded":"2023-10-30T12:45:20.377142+0000","last_fullsized":"2023-10-30T12:45:20.377142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:33:03.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470206+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:20.470206+0000","last_peered":"2023-10-30T12:45:20.470206+0000","last_clean":"2023-10-30T12:45:20.470206+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:20.470206+0000","last_undegraded":"2023-10-30T12:45:20.470206+0000","last_fullsized":"2023-10-30T12:45:20.470206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:31:49.660950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376996+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:20.376996+0000","last_peered":"2023-10-30T12:45:20.376996+0000","last_clean":"2023-10-30T12:45:20.376996+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:20.376996+0000","last_undegraded":"2023-10-30T12:45:20.376996+0000","last_fullsized":"2023-10-30T12:45:20.376996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:51.286544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471712+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:20.471712+0000","last_peered":"2023-10-30T12:45:20.471712+0000","last_clean":"2023-10-30T12:45:20.471712+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:20.471712+0000","last_undegraded":"2023-10-30T12:45:20.471712+0000","last_fullsized":"2023-10-30T12:45:20.471712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:57.636148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471671+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:20.471671+0000","last_peered":"2023-10-30T12:45:20.471671+0000","last_clean":"2023-10-30T12:45:20.471671+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:20.471671+0000","last_undegraded":"2023-10-30T12:45:20.471671+0000","last_fullsized":"2023-10-30T12:45:20.471671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:30.410457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067039+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:20.067039+0000","last_peered":"2023-10-30T12:45:20.067039+0000","last_clean":"2023-10-30T12:45:20.067039+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:20.067039+0000","last_undegraded":"2023-10-30T12:45:20.067039+0000","last_fullsized":"2023-10-30T12:45:20.067039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:46.788124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471494+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:20.471494+0000","last_peered":"2023-10-30T12:45:20.471494+0000","last_clean":"2023-10-30T12:45:20.471494+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:20.471494+0000","last_undegraded":"2023-10-30T12:45:20.471494+0000","last_fullsized":"2023-10-30T12:45:20.471494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:44.555138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471084+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:20.471084+0000","last_peered":"2023-10-30T12:45:20.471084+0000","last_clean":"2023-10-30T12:45:20.471084+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:20.471084+0000","last_undegraded":"2023-10-30T12:45:20.471084+0000","last_fullsized":"2023-10-30T12:45:20.471084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:29.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377430+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:20.377430+0000","last_peered":"2023-10-30T12:45:20.377430+0000","last_clean":"2023-10-30T12:45:20.377430+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:20.377430+0000","last_undegraded":"2023-10-30T12:45:20.377430+0000","last_fullsized":"2023-10-30T12:45:20.377430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:50:53.563609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471353+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:20.471353+0000","last_peered":"2023-10-30T12:45:20.471353+0000","last_clean":"2023-10-30T12:45:20.471353+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:20.471353+0000","last_undegraded":"2023-10-30T12:45:20.471353+0000","last_fullsized":"2023-10-30T12:45:20.471353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:12:59.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470912+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:20.470912+0000","last_peered":"2023-10-30T12:45:20.470912+0000","last_clean":"2023-10-30T12:45:20.470912+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:20.470912+0000","last_undegraded":"2023-10-30T12:45:20.470912+0000","last_fullsized":"2023-10-30T12:45:20.470912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:10.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071561+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:20.071561+0000","last_peered":"2023-10-30T12:45:20.071561+0000","last_clean":"2023-10-30T12:45:20.071561+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:20.071561+0000","last_undegraded":"2023-10-30T12:45:20.071561+0000","last_fullsized":"2023-10-30T12:45:20.071561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:17:56.183441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471094+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:20.471094+0000","last_peered":"2023-10-30T12:45:20.471094+0000","last_clean":"2023-10-30T12:45:20.471094+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:20.471094+0000","last_undegraded":"2023-10-30T12:45:20.471094+0000","last_fullsized":"2023-10-30T12:45:20.471094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471141+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:20.471141+0000","last_peered":"2023-10-30T12:45:20.471141+0000","last_clean":"2023-10-30T12:45:20.471141+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:20.471141+0000","last_undegraded":"2023-10-30T12:45:20.471141+0000","last_fullsized":"2023-10-30T12:45:20.471141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:23:00.831740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630920+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:19.630920+0000","last_peered":"2023-10-30T12:45:19.630920+0000","last_clean":"2023-10-30T12:45:19.630920+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:19.630920+0000","last_undegraded":"2023-10-30T12:45:19.630920+0000","last_fullsized":"2023-10-30T12:45:19.630920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470208+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:20.470208+0000","last_peered":"2023-10-30T12:45:20.470208+0000","last_clean":"2023-10-30T12:45:20.470208+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:20.470208+0000","last_undegraded":"2023-10-30T12:45:20.470208+0000","last_fullsized":"2023-10-30T12:45:20.470208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:26.174484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633996+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:19.633996+0000","last_peered":"2023-10-30T12:45:19.633996+0000","last_clean":"2023-10-30T12:45:19.633996+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:19.633996+0000","last_undegraded":"2023-10-30T12:45:19.633996+0000","last_fullsized":"2023-10-30T12:45:19.633996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:12:12.444137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633919+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:19.633919+0000","last_peered":"2023-10-30T12:45:19.633919+0000","last_clean":"2023-10-30T12:45:19.633919+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:19.633919+0000","last_undegraded":"2023-10-30T12:45:19.633919+0000","last_fullsized":"2023-10-30T12:45:19.633919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377703+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:20.377703+0000","last_peered":"2023-10-30T12:45:20.377703+0000","last_clean":"2023-10-30T12:45:20.377703+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:20.377703+0000","last_undegraded":"2023-10-30T12:45:20.377703+0000","last_fullsized":"2023-10-30T12:45:20.377703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:40:41.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634489+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:19.634489+0000","last_peered":"2023-10-30T12:45:19.634489+0000","last_clean":"2023-10-30T12:45:19.634489+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:19.634489+0000","last_undegraded":"2023-10-30T12:45:19.634489+0000","last_fullsized":"2023-10-30T12:45:19.634489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471307+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:20.471307+0000","last_peered":"2023-10-30T12:45:20.471307+0000","last_clean":"2023-10-30T12:45:20.471307+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:20.471307+0000","last_undegraded":"2023-10-30T12:45:20.471307+0000","last_fullsized":"2023-10-30T12:45:20.471307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:07:09.393386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.081621+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:24.081621+0000","last_peered":"2023-10-30T12:45:24.081621+0000","last_clean":"2023-10-30T12:45:24.081621+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:24.081621+0000","last_undegraded":"2023-10-30T12:45:24.081621+0000","last_fullsized":"2023-10-30T12:45:24.081621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072148+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:20.072148+0000","last_peered":"2023-10-30T12:45:20.072148+0000","last_clean":"2023-10-30T12:45:20.072148+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:20.072148+0000","last_undegraded":"2023-10-30T12:45:20.072148+0000","last_fullsized":"2023-10-30T12:45:20.072148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:17.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.085227+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:24.085227+0000","last_peered":"2023-10-30T12:45:24.085227+0000","last_clean":"2023-10-30T12:45:24.085227+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:24.085227+0000","last_undegraded":"2023-10-30T12:45:24.085227+0000","last_fullsized":"2023-10-30T12:45:24.085227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069876+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:20.069876+0000","last_peered":"2023-10-30T12:45:20.069876+0000","last_clean":"2023-10-30T12:45:20.069876+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:20.069876+0000","last_undegraded":"2023-10-30T12:45:20.069876+0000","last_fullsized":"2023-10-30T12:45:20.069876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:07:42.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.083457+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:24.083457+0000","last_peered":"2023-10-30T12:45:24.083457+0000","last_clean":"2023-10-30T12:45:24.083457+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:24.083457+0000","last_undegraded":"2023-10-30T12:45:24.083457+0000","last_fullsized":"2023-10-30T12:45:24.083457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470757+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:20.470757+0000","last_peered":"2023-10-30T12:45:20.470757+0000","last_clean":"2023-10-30T12:45:20.470757+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:20.470757+0000","last_undegraded":"2023-10-30T12:45:20.470757+0000","last_fullsized":"2023-10-30T12:45:20.470757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:17.831426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634163+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:19.634163+0000","last_peered":"2023-10-30T12:45:19.634163+0000","last_clean":"2023-10-30T12:45:19.634163+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:19.634163+0000","last_undegraded":"2023-10-30T12:45:19.634163+0000","last_fullsized":"2023-10-30T12:45:19.634163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070431+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:20.070431+0000","last_peered":"2023-10-30T12:45:20.070431+0000","last_clean":"2023-10-30T12:45:20.070431+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:20.070431+0000","last_undegraded":"2023-10-30T12:45:20.070431+0000","last_fullsized":"2023-10-30T12:45:20.070431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:14:31.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.633547+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:19.633547+0000","last_peered":"2023-10-30T12:45:19.633547+0000","last_clean":"2023-10-30T12:45:19.633547+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:19.633547+0000","last_undegraded":"2023-10-30T12:45:19.633547+0000","last_fullsized":"2023-10-30T12:45:19.633547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470957+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:20.470957+0000","last_peered":"2023-10-30T12:45:20.470957+0000","last_clean":"2023-10-30T12:45:20.470957+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:20.470957+0000","last_undegraded":"2023-10-30T12:45:20.470957+0000","last_fullsized":"2023-10-30T12:45:20.470957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:13.282854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.080427+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:24.080427+0000","last_peered":"2023-10-30T12:45:24.080427+0000","last_clean":"2023-10-30T12:45:24.080427+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:24.080427+0000","last_undegraded":"2023-10-30T12:45:24.080427+0000","last_fullsized":"2023-10-30T12:45:24.080427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069448+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:20.069448+0000","last_peered":"2023-10-30T12:45:20.069448+0000","last_clean":"2023-10-30T12:45:20.069448+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:20.069448+0000","last_undegraded":"2023-10-30T12:45:20.069448+0000","last_fullsized":"2023-10-30T12:45:20.069448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:14.563685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.073220+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:20.073220+0000","last_peered":"2023-10-30T12:45:20.073220+0000","last_clean":"2023-10-30T12:45:20.073220+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:20.073220+0000","last_undegraded":"2023-10-30T12:45:20.073220+0000","last_fullsized":"2023-10-30T12:45:20.073220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071276+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:20.071276+0000","last_peered":"2023-10-30T12:45:20.071276+0000","last_clean":"2023-10-30T12:45:20.071276+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:20.071276+0000","last_undegraded":"2023-10-30T12:45:20.071276+0000","last_fullsized":"2023-10-30T12:45:20.071276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:43.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.077908+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:24.077908+0000","last_peered":"2023-10-30T12:45:24.077908+0000","last_clean":"2023-10-30T12:45:24.077908+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:24.077908+0000","last_undegraded":"2023-10-30T12:45:24.077908+0000","last_fullsized":"2023-10-30T12:45:24.077908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072543+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:20.072543+0000","last_peered":"2023-10-30T12:45:20.072543+0000","last_clean":"2023-10-30T12:45:20.072543+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:20.072543+0000","last_undegraded":"2023-10-30T12:45:20.072543+0000","last_fullsized":"2023-10-30T12:45:20.072543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:54.641020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067271+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:20.067271+0000","last_peered":"2023-10-30T12:45:20.067271+0000","last_clean":"2023-10-30T12:45:20.067271+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:20.067271+0000","last_undegraded":"2023-10-30T12:45:20.067271+0000","last_fullsized":"2023-10-30T12:45:20.067271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378262+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:20.378262+0000","last_peered":"2023-10-30T12:45:20.378262+0000","last_clean":"2023-10-30T12:45:20.378262+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:20.378262+0000","last_undegraded":"2023-10-30T12:45:20.378262+0000","last_fullsized":"2023-10-30T12:45:20.378262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:08.395587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630441+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:19.630441+0000","last_peered":"2023-10-30T12:45:19.630441+0000","last_clean":"2023-10-30T12:45:19.630441+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:19.630441+0000","last_undegraded":"2023-10-30T12:45:19.630441+0000","last_fullsized":"2023-10-30T12:45:19.630441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069897+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:20.069897+0000","last_peered":"2023-10-30T12:45:20.069897+0000","last_clean":"2023-10-30T12:45:20.069897+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:20.069897+0000","last_undegraded":"2023-10-30T12:45:20.069897+0000","last_fullsized":"2023-10-30T12:45:20.069897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:58:00.569432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071157+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:19.071157+0000","last_peered":"2023-10-30T12:45:19.071157+0000","last_clean":"2023-10-30T12:45:19.071157+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:19.071157+0000","last_undegraded":"2023-10-30T12:45:19.071157+0000","last_fullsized":"2023-10-30T12:45:19.071157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634249+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:19.634249+0000","last_peered":"2023-10-30T12:45:19.634249+0000","last_clean":"2023-10-30T12:45:19.634249+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:19.634249+0000","last_undegraded":"2023-10-30T12:45:19.634249+0000","last_fullsized":"2023-10-30T12:45:19.634249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470630+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:20.470630+0000","last_peered":"2023-10-30T12:45:20.470630+0000","last_clean":"2023-10-30T12:45:20.470630+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:20.470630+0000","last_undegraded":"2023-10-30T12:45:20.470630+0000","last_fullsized":"2023-10-30T12:45:20.470630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:25:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634164+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:19.634164+0000","last_peered":"2023-10-30T12:45:19.634164+0000","last_clean":"2023-10-30T12:45:19.634164+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:19.634164+0000","last_undegraded":"2023-10-30T12:45:19.634164+0000","last_fullsized":"2023-10-30T12:45:19.634164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470604+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:20.470604+0000","last_peered":"2023-10-30T12:45:20.470604+0000","last_clean":"2023-10-30T12:45:20.470604+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:20.470604+0000","last_undegraded":"2023-10-30T12:45:20.470604+0000","last_fullsized":"2023-10-30T12:45:20.470604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:55.822788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630426+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:19.630426+0000","last_peered":"2023-10-30T12:45:19.630426+0000","last_clean":"2023-10-30T12:45:19.630426+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:19.630426+0000","last_undegraded":"2023-10-30T12:45:19.630426+0000","last_fullsized":"2023-10-30T12:45:19.630426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066971+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:20.066971+0000","last_peered":"2023-10-30T12:45:20.066971+0000","last_clean":"2023-10-30T12:45:20.066971+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:20.066971+0000","last_undegraded":"2023-10-30T12:45:20.066971+0000","last_fullsized":"2023-10-30T12:45:20.066971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:17:16.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634197+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:19.634197+0000","last_peered":"2023-10-30T12:45:19.634197+0000","last_clean":"2023-10-30T12:45:19.634197+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:19.634197+0000","last_undegraded":"2023-10-30T12:45:19.634197+0000","last_fullsized":"2023-10-30T12:45:19.634197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072149+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:20.072149+0000","last_peered":"2023-10-30T12:45:20.072149+0000","last_clean":"2023-10-30T12:45:20.072149+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:20.072149+0000","last_undegraded":"2023-10-30T12:45:20.072149+0000","last_fullsized":"2023-10-30T12:45:20.072149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630601+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.630601+0000","last_peered":"2023-10-30T12:45:19.630601+0000","last_clean":"2023-10-30T12:45:19.630601+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.630601+0000","last_undegraded":"2023-10-30T12:45:19.630601+0000","last_fullsized":"2023-10-30T12:45:19.630601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378093+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:20.378093+0000","last_peered":"2023-10-30T12:45:20.378093+0000","last_clean":"2023-10-30T12:45:20.378093+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:20.378093+0000","last_undegraded":"2023-10-30T12:45:20.378093+0000","last_fullsized":"2023-10-30T12:45:20.378093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:05.357577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634481+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:19.634481+0000","last_peered":"2023-10-30T12:45:19.634481+0000","last_clean":"2023-10-30T12:45:19.634481+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:19.634481+0000","last_undegraded":"2023-10-30T12:45:19.634481+0000","last_fullsized":"2023-10-30T12:45:19.634481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066974+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:20.066974+0000","last_peered":"2023-10-30T12:45:20.066974+0000","last_clean":"2023-10-30T12:45:20.066974+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:20.066974+0000","last_undegraded":"2023-10-30T12:45:20.066974+0000","last_fullsized":"2023-10-30T12:45:20.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:46:43.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630684+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:19.630684+0000","last_peered":"2023-10-30T12:45:19.630684+0000","last_clean":"2023-10-30T12:45:19.630684+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:19.630684+0000","last_undegraded":"2023-10-30T12:45:19.630684+0000","last_fullsized":"2023-10-30T12:45:19.630684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471700+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:20.471700+0000","last_peered":"2023-10-30T12:45:20.471700+0000","last_clean":"2023-10-30T12:45:20.471700+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:20.471700+0000","last_undegraded":"2023-10-30T12:45:20.471700+0000","last_fullsized":"2023-10-30T12:45:20.471700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:25.320280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.082644+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:24.082644+0000","last_peered":"2023-10-30T12:45:24.082644+0000","last_clean":"2023-10-30T12:45:24.082644+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:24.082644+0000","last_undegraded":"2023-10-30T12:45:24.082644+0000","last_fullsized":"2023-10-30T12:45:24.082644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471488+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:20.471488+0000","last_peered":"2023-10-30T12:45:20.471488+0000","last_clean":"2023-10-30T12:45:20.471488+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:20.471488+0000","last_undegraded":"2023-10-30T12:45:20.471488+0000","last_fullsized":"2023-10-30T12:45:20.471488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:14.407746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.634110+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:19.634110+0000","last_peered":"2023-10-30T12:45:19.634110+0000","last_clean":"2023-10-30T12:45:19.634110+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:19.634110+0000","last_undegraded":"2023-10-30T12:45:19.634110+0000","last_fullsized":"2023-10-30T12:45:19.634110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071773+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:20.071773+0000","last_peered":"2023-10-30T12:45:20.071773+0000","last_clean":"2023-10-30T12:45:20.071773+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:20.071773+0000","last_undegraded":"2023-10-30T12:45:20.071773+0000","last_fullsized":"2023-10-30T12:45:20.071773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:09.037055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630574+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:19.630574+0000","last_peered":"2023-10-30T12:45:19.630574+0000","last_clean":"2023-10-30T12:45:19.630574+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:19.630574+0000","last_undegraded":"2023-10-30T12:45:19.630574+0000","last_fullsized":"2023-10-30T12:45:19.630574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470363+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:20.470363+0000","last_peered":"2023-10-30T12:45:20.470363+0000","last_clean":"2023-10-30T12:45:20.470363+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:20.470363+0000","last_undegraded":"2023-10-30T12:45:20.470363+0000","last_fullsized":"2023-10-30T12:45:20.470363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:45:36.069824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630584+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:19.630584+0000","last_peered":"2023-10-30T12:45:19.630584+0000","last_clean":"2023-10-30T12:45:19.630584+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:19.630584+0000","last_undegraded":"2023-10-30T12:45:19.630584+0000","last_fullsized":"2023-10-30T12:45:19.630584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:21.343385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084421+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:24.084421+0000","last_peered":"2023-10-30T12:45:24.084421+0000","last_clean":"2023-10-30T12:45:24.084421+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:24.084421+0000","last_undegraded":"2023-10-30T12:45:24.084421+0000","last_fullsized":"2023-10-30T12:45:24.084421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470979+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:20.470979+0000","last_peered":"2023-10-30T12:45:20.470979+0000","last_clean":"2023-10-30T12:45:20.470979+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:20.470979+0000","last_undegraded":"2023-10-30T12:45:20.470979+0000","last_fullsized":"2023-10-30T12:45:20.470979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:44.769458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377742+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:20.377742+0000","last_peered":"2023-10-30T12:45:20.377742+0000","last_clean":"2023-10-30T12:45:20.377742+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:20.377742+0000","last_undegraded":"2023-10-30T12:45:20.377742+0000","last_fullsized":"2023-10-30T12:45:20.377742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:13.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084274+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:24.084274+0000","last_peered":"2023-10-30T12:45:24.084274+0000","last_clean":"2023-10-30T12:45:24.084274+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:24.084274+0000","last_undegraded":"2023-10-30T12:45:24.084274+0000","last_fullsized":"2023-10-30T12:45:24.084274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377958+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:20.377958+0000","last_peered":"2023-10-30T12:45:20.377958+0000","last_clean":"2023-10-30T12:45:20.377958+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:20.377958+0000","last_undegraded":"2023-10-30T12:45:20.377958+0000","last_fullsized":"2023-10-30T12:45:20.377958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:08.137679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.085394+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:24.085394+0000","last_peered":"2023-10-30T12:45:24.085394+0000","last_clean":"2023-10-30T12:45:24.085394+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:24.085394+0000","last_undegraded":"2023-10-30T12:45:24.085394+0000","last_fullsized":"2023-10-30T12:45:24.085394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471292+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:20.471292+0000","last_peered":"2023-10-30T12:45:20.471292+0000","last_clean":"2023-10-30T12:45:20.471292+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:20.471292+0000","last_undegraded":"2023-10-30T12:45:20.471292+0000","last_fullsized":"2023-10-30T12:45:20.471292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:26.117300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471675+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:20.471675+0000","last_peered":"2023-10-30T12:45:20.471675+0000","last_clean":"2023-10-30T12:45:20.471675+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:20.471675+0000","last_undegraded":"2023-10-30T12:45:20.471675+0000","last_fullsized":"2023-10-30T12:45:20.471675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471526+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:20.471526+0000","last_peered":"2023-10-30T12:45:20.471526+0000","last_clean":"2023-10-30T12:45:20.471526+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:20.471526+0000","last_undegraded":"2023-10-30T12:45:20.471526+0000","last_fullsized":"2023-10-30T12:45:20.471526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:57:18.999410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.082716+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:24.082716+0000","last_peered":"2023-10-30T12:45:24.082716+0000","last_clean":"2023-10-30T12:45:24.082716+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:24.082716+0000","last_undegraded":"2023-10-30T12:45:24.082716+0000","last_fullsized":"2023-10-30T12:45:24.082716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7919999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.964}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":806912,"data_stored":660885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:30.387 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T12:45:30.387 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-30T12:45:30.388 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-30T12:45:30.388 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-30T12:45:30.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 Processor -- start 2023-10-30T12:45:30.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 -- start start 2023-10-30T12:45:30.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f494806a0f0 0x7f4948159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49481537f0 0x7f4948153c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480cfe40 con 0x7f49480c9ad0 2023-10-30T12:45:30.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480ca010 con 0x7f494806a0f0 2023-10-30T12:45:30.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494fd9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480ca2b0 con 0x7f49481537f0 2023-10-30T12:45:30.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494db39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f494806a0f0 0x7f4948159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494e33a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49481537f0 0x7f4948153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494d338640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f49480c9ad0 0x7f494806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45114/0 (socket says 172.21.15.8:45114) 2023-10-30T12:45:30.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494d338640 1 -- 172.21.15.8:0/2137245165 learned_addr learned my addr 172.21.15.8:0/2137245165 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:30.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494db39640 1 -- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49481537f0 msgr2=0x7f4948153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494db39640 1 --2- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49481537f0 0x7f4948153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494db39640 1 -- 172.21.15.8:0/2137245165 >> v1:172.21.15.46:6789/0 conn(0x7f49480c9ad0 legacy=0x7f494806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.569+0000 7f494db39640 1 -- 172.21.15.8:0/2137245165 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49481541c0 con 0x7f494806a0f0 2023-10-30T12:45:30.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494db39640 1 --2- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494806a0f0 0x7f4948159a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49440097c0 tx=0x7f494401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00441e6cef7a7a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494cb37640 1 -- 172.21.15.8:0/2137245165 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440a8970 con 0x7f494806a0f0 2023-10-30T12:45:30.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494cb37640 1 -- 172.21.15.8:0/2137245165 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49440a8ad0 con 0x7f494806a0f0 2023-10-30T12:45:30.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494cb37640 1 -- 172.21.15.8:0/2137245165 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440a8e60 con 0x7f494806a0f0 2023-10-30T12:45:30.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 -- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494806a0f0 msgr2=0x7f4948159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 --2- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494806a0f0 0x7f4948159a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49440097c0 tx=0x7f494401ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:30.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 -- 172.21.15.8:0/2137245165 shutdown_connections 2023-10-30T12:45:30.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 --2- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49481537f0 0x7f4948153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 --2- 172.21.15.8:0/2137245165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494806a0f0 0x7f4948159a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 -- 172.21.15.8:0/2137245165 >> 172.21.15.8:0/2137245165 conn(0x7f49480cf150 msgr2=0x7f49480cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:30.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 -- 172.21.15.8:0/2137245165 shutdown_connections 2023-10-30T12:45:30.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.573+0000 7f494fd9d640 1 -- 172.21.15.8:0/2137245165 wait complete. 2023-10-30T12:45:30.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 Processor -- start 2023-10-30T12:45:30.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- start start 2023-10-30T12:45:30.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f494806a0f0 0x7f49480b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49480c9ad0 0x7f49481f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481e4540 con 0x7f49481537f0 2023-10-30T12:45:30.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481e34b0 con 0x7f49480c9ad0 2023-10-30T12:45:30.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49481e5dc0 con 0x7f494806a0f0 2023-10-30T12:45:30.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55942/0 (socket says 172.21.15.8:55942) 2023-10-30T12:45:30.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 -- 172.21.15.8:0/511223995 learned_addr learned my addr 172.21.15.8:0/511223995 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:30.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494db39640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f494806a0f0 0x7f49480b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494d338640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49480c9ad0 0x7f49481f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 -- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f494806a0f0 msgr2=0x7f49480b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f494806a0f0 0x7f49480b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 -- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49480c9ad0 msgr2=0x7f49481f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49480c9ad0 0x7f49481f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 -- 172.21.15.8:0/511223995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009420 con 0x7f49481537f0 2023-10-30T12:45:30.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494e33a640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4940009d30 tx=0x7f494000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940009ba0 con 0x7f49481537f0 2023-10-30T12:45:30.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000f040 con 0x7f49481537f0 2023-10-30T12:45:30.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400146c0 con 0x7f49481537f0 2023-10-30T12:45:30.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49480caea0 con 0x7f49481537f0 2023-10-30T12:45:30.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.577+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49481e2ef0 con 0x7f49481537f0 2023-10-30T12:45:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804e170 con 0x7f49481537f0 2023-10-30T12:45:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4940014820 con 0x7f49481537f0 2023-10-30T12:45:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f493e7fc640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 0x7f492c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4940084a70 con 0x7f49481537f0 2023-10-30T12:45:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f494db39640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 0x7f492c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.581+0000 7f494db39640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 0x7f492c068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49440b2000 tx=0x7f4944008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.585+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4940052030 con 0x7f49481537f0 2023-10-30T12:45:30.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.777+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f494814dae0 con 0x7f49481537f0 2023-10-30T12:45:30.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.777+0000 7f493e7fc640 1 -- 172.21.15.8:0/511223995 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4940018040 con 0x7f49481537f0 2023-10-30T12:45:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:30.782 INFO:teuthology.orchestra.run.smithi008.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-30T12:45:30.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 msgr2=0x7f492c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 0x7f492c068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49440b2000 tx=0x7f4944008040 comp rx=0 tx=0).stop 2023-10-30T12:45:30.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 msgr2=0x7f49481f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4940009d30 tx=0x7f494000c490 comp rx=0 tx=0).stop 2023-10-30T12:45:30.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 shutdown_connections 2023-10-30T12:45:30.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f492c0664a0 0x7f492c068960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49480c9ad0 0x7f49481f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49481537f0 0x7f49481f3230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 --2- 172.21.15.8:0/511223995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f494806a0f0 0x7f49480b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 >> 172.21.15.8:0/511223995 conn(0x7f49480cf150 msgr2=0x7f4948151a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:30.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 shutdown_connections 2023-10-30T12:45:30.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.781+0000 7f494fd9d640 1 -- 172.21.15.8:0/511223995 wait complete. 2023-10-30T12:45:30.803 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-30T12:45:30.803 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:30.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.941+0000 7fab4d70f640 1 Processor -- start 2023-10-30T12:45:30.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab4d70f640 1 -- start start 2023-10-30T12:45:30.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab4d70f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab48159ec0 0x7fab480c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab4d70f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4815cda0 con 0x7fab480c5ba0 2023-10-30T12:45:30.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab4d70f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4806f490 con 0x7fab480c5f80 2023-10-30T12:45:30.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab4d70f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815d990 con 0x7fab48159ec0 2023-10-30T12:45:30.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab46ffd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fab480c5ba0 0x7fab48069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53902/0 (socket says 172.21.15.8:53902) 2023-10-30T12:45:30.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab46ffd640 1 -- 172.21.15.8:0/248205633 learned_addr learned my addr 172.21.15.8:0/248205633 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:30.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab477fe640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab48159ec0 0x7fab480c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825743149 0 0) 0x7fab4815cda0 con 0x7fab480c5ba0 2023-10-30T12:45:30.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab34003530 con 0x7fab480c5ba0 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902961127 0 0) 0x7fab34003530 con 0x7fab480c5ba0 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab48159ec0 msgr2=0x7fab480c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab48159ec0 0x7fab480c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 >> v1:172.21.15.46:6789/0 conn(0x7fab480c5f80 legacy=0x7fab481597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4815cff0 con 0x7fab480c5ba0 2023-10-30T12:45:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fab38001680 con 0x7fab480c5ba0 2023-10-30T12:45:30.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 >> v1:172.21.15.8:6789/0 conn(0x7fab480c5ba0 legacy=0x7fab48069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab340039b0 0x7fab34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab34004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab34005030 0x7fab3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815d990 con 0x7fab34004360 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34005670 con 0x7fab340039b0 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3400bf20 con 0x7fab34005030 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab34004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab46ffd640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab340039b0 0x7fab34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab477fe640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab34005030 0x7fab3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 -- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab34005030 msgr2=0x7fab3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab34005030 0x7fab3400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 -- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab340039b0 msgr2=0x7fab34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab340039b0 0x7fab34003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab3ffff640 1 -- 172.21.15.8:0/248205633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab34003530 con 0x7fab34004360 2023-10-30T12:45:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.945+0000 7fab46ffd640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab340039b0 0x7fab34003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab3ffff640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab34004830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab4806f0e0 tx=0x7fab3001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319c795ce9566212 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300a8830 con 0x7fab34004360 2023-10-30T12:45:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab300a8990 con 0x7fab34004360 2023-10-30T12:45:30.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab467fc640 1 -- 172.21.15.8:0/248205633 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300a8d40 con 0x7fab34004360 2023-10-30T12:45:30.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 -- 172.21.15.8:0/248205633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 msgr2=0x7fab34004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 --2- 172.21.15.8:0/248205633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab34004830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab4806f0e0 tx=0x7fab3001ec10 comp rx=0 tx=0).stop 2023-10-30T12:45:30.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab477fe640 1 -- 172.21.15.8:0/248205633 reap_dead start 2023-10-30T12:45:30.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 -- 172.21.15.8:0/248205633 shutdown_connections 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 -- 172.21.15.8:0/248205633 >> 172.21.15.8:0/248205633 conn(0x7fab4806afa0 msgr2=0x7fab4806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 -- 172.21.15.8:0/248205633 shutdown_connections 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.949+0000 7fab4d70f640 1 -- 172.21.15.8:0/248205633 wait complete. 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 Processor -- start 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- start start 2023-10-30T12:45:30.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab481e2de0 0x7fab481e3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab481e3c60 0x7fab480c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48071d20 con 0x7fab34004360 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48071f10 con 0x7fab481e2de0 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab481f1520 con 0x7fab481e3c60 2023-10-30T12:45:30.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55952/0 (socket says 172.21.15.8:55952) 2023-10-30T12:45:30.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 -- 172.21.15.8:0/1583842099 learned_addr learned my addr 172.21.15.8:0/1583842099 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:30.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab481e2de0 0x7fab481e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab477fe640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab481e3c60 0x7fab480c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 -- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab481e3c60 msgr2=0x7fab480c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab481e3c60 0x7fab480c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 -- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab481e2de0 msgr2=0x7fab481e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:30.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab481e2de0 0x7fab481e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:30.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 -- 172.21.15.8:0/1583842099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab481f17e0 con 0x7fab34004360 2023-10-30T12:45:30.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab46ffd640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e7b0 tx=0x7fab3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800eec0 con 0x7fab34004360 2023-10-30T12:45:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800c870 con 0x7fab34004360 2023-10-30T12:45:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab481ee0c0 con 0x7fab34004360 2023-10-30T12:45:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800cc00 con 0x7fab34004360 2023-10-30T12:45:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481ee680 con 0x7fab34004360 2023-10-30T12:45:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.953+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4804e170 con 0x7fab34004360 2023-10-30T12:45:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.957+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fab3801d460 con 0x7fab34004360 2023-10-30T12:45:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.957+0000 7fab3f7fe640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 0x7fab28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.957+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab38083fb0 con 0x7fab34004360 2023-10-30T12:45:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.957+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 0x7fab28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.957+0000 7fab3ffff640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 0x7fab28068960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab300b2000 tx=0x7fab300aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:30.961+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fab38051300 con 0x7fab34004360 2023-10-30T12:45:31.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab481535b0 con 0x7fab34004360 2023-10-30T12:45:31.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab3f7fe640 1 -- 172.21.15.8:0/1583842099 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7fab38054920 con 0x7fab34004360 2023-10-30T12:45:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:31.128 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:31.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 msgr2=0x7fab28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 0x7fab28068960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab300b2000 tx=0x7fab300aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:31.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 msgr2=0x7fab4814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e7b0 tx=0x7fab3800ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:31.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 shutdown_connections 2023-10-30T12:45:31.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fab280664a0 0x7fab28068960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab481e2de0 0x7fab481e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fab481e3c60 0x7fab480c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 --2- 172.21.15.8:0/1583842099 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab34004360 0x7fab4814e900 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.125+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 >> 172.21.15.8:0/1583842099 conn(0x7fab4806afa0 msgr2=0x7fab4814b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.129+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 shutdown_connections 2023-10-30T12:45:31.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.129+0000 7fab4d70f640 1 -- 172.21.15.8:0/1583842099 wait complete. 2023-10-30T12:45:31.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:31.150 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:31.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 Processor -- start 2023-10-30T12:45:31.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 -- start start 2023-10-30T12:45:31.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 0x7fdcfc0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdcfc0c4410 0x7fdcfc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc153f80 con 0x7fdcfc0c4810 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc159df0 con 0x7fdcfc0c4410 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd03cd6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15a7c0 con 0x7fdcfc153ba0 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd01271640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdcfc0c4410 0x7fdcfc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd01271640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdcfc0c4410 0x7fdcfc15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55968/0 (socket says 172.21.15.8:55968) 2023-10-30T12:45:31.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.293+0000 7fdd01271640 1 -- 172.21.15.8:0/2204732643 learned_addr learned my addr 172.21.15.8:0/2204732643 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01a72640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 0x7fdcfc0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01271640 1 -- 172.21.15.8:0/2204732643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 msgr2=0x7fdcfc0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01271640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 0x7fdcfc0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01271640 1 -- 172.21.15.8:0/2204732643 >> v1:172.21.15.46:6789/0 conn(0x7fdcfc0c4810 legacy=0x7fdcfc15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01271640 1 -- 172.21.15.8:0/2204732643 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc1597c0 con 0x7fdcfc0c4410 2023-10-30T12:45:31.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01a72640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 0x7fdcfc0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd01271640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4410 0x7fdcfc15a280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009870 tx=0x7fdcec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57eba3d4ff31c3b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd00a70640 1 -- 172.21.15.8:0/2204732643 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0ab680 con 0x7fdcfc0c4410 2023-10-30T12:45:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd00a70640 1 -- 172.21.15.8:0/2204732643 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0ab7e0 con 0x7fdcfc0c4410 2023-10-30T12:45:31.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd00a70640 1 -- 172.21.15.8:0/2204732643 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0abb70 con 0x7fdcfc0c4410 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 -- 172.21.15.8:0/2204732643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4410 msgr2=0x7fdcfc15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4410 0x7fdcfc15a280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009870 tx=0x7fdcec01c480 comp rx=0 tx=0).stop 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 -- 172.21.15.8:0/2204732643 shutdown_connections 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4410 0x7fdcfc15a280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/2204732643 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc153ba0 0x7fdcfc0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.297+0000 7fdd03cd6640 1 -- 172.21.15.8:0/2204732643 >> 172.21.15.8:0/2204732643 conn(0x7fdcfc071790 msgr2=0x7fdcfc071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- 172.21.15.8:0/2204732643 shutdown_connections 2023-10-30T12:45:31.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- 172.21.15.8:0/2204732643 wait complete. 2023-10-30T12:45:31.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 Processor -- start 2023-10-30T12:45:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- start start 2023-10-30T12:45:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 0x7fdcfc1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 0x7fdcfc14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b3c80 con 0x7fdcfc0c4810 2023-10-30T12:45:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b5120 con 0x7fdcfc153ba0 2023-10-30T12:45:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd03cd6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b3700 con 0x7fdcfc0c4410 2023-10-30T12:45:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55972/0 (socket says 172.21.15.8:55972) 2023-10-30T12:45:31.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 -- 172.21.15.8:0/1413695 learned_addr learned my addr 172.21.15.8:0/1413695 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd02273640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 0x7fdcfc14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01a72640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 0x7fdcfc1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 -- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 msgr2=0x7fdcfc1f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 0x7fdcfc1f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 -- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 msgr2=0x7fdcfc14e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 0x7fdcfc14e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 -- 172.21.15.8:0/1413695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0094d0 con 0x7fdcfc0c4810 2023-10-30T12:45:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01a72640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 0x7fdcfc1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd02273640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 0x7fdcfc14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdd01271640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcec01a0b0 tx=0x7fdcec01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0b4800 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0b4960 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.301+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0b4d10 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc0b4b60 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc0b39a0 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc04e170 con 0x7fdcfc0c4810 2023-10-30T12:45:31.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdcec0b3380 con 0x7fdcfc0c4810 2023-10-30T12:45:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdcf27fc640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 0x7fdce0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcec11e8e0 con 0x7fdcfc0c4810 2023-10-30T12:45:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdd01a72640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 0x7fdce0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.305+0000 7fdd01a72640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 0x7fdce0068990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b990 tx=0x7fdcf8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.309+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdcec0ebd70 con 0x7fdcfc0c4810 2023-10-30T12:45:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.473+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcfc0cabb0 con 0x7fdcfc0c4810 2023-10-30T12:45:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.473+0000 7fdcf27fc640 1 -- 172.21.15.8:0/1413695 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcec0c40a0 con 0x7fdcfc0c4810 2023-10-30T12:45:31.477 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 msgr2=0x7fdce0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 0x7fdce0068990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b990 tx=0x7fdcf8008040 comp rx=0 tx=0).stop 2023-10-30T12:45:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 msgr2=0x7fdcfc0ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcec01a0b0 tx=0x7fdcec01acc0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 shutdown_connections 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdce00664d0 0x7fdce0068990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcfc153ba0 0x7fdcfc14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdcfc0c4810 0x7fdcfc0ce2a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 --2- 172.21.15.8:0/1413695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdcfc0c4410 0x7fdcfc1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 >> 172.21.15.8:0/1413695 conn(0x7fdcfc071790 msgr2=0x7fdcfc158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 shutdown_connections 2023-10-30T12:45:31.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.477+0000 7fdd03cd6640 1 -- 172.21.15.8:0/1413695 wait complete. 2023-10-30T12:45:31.498 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:45:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 Processor -- start 2023-10-30T12:45:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 -- start start 2023-10-30T12:45:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f53381577d0 0x7f5338157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53380b2560 con 0x7f5338071820 2023-10-30T12:45:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53380b2c60 con 0x7f5338071c00 2023-10-30T12:45:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533d409640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5338158160 con 0x7f53381577d0 2023-10-30T12:45:31.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f5337fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f5338071820 0x7f533815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53906/0 (socket says 172.21.15.8:53906) 2023-10-30T12:45:31.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f5337fff640 1 -- 172.21.15.8:0/3444801927 learned_addr learned my addr 172.21.15.8:0/3444801927 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:31.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533cc08640 1 --2- 172.21.15.8:0/3444801927 >> v2:172.21.15.8:3300/0 conn(0x7f53381577d0 0x7f5338157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f5336ffd640 1 -- 172.21.15.8:0/3444801927 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871874421 0 0) 0x7f53380b2560 con 0x7f5338071820 2023-10-30T12:45:31.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533cc08640 1 -- 172.21.15.8:0/3444801927 >> v1:172.21.15.46:6789/0 conn(0x7f5338071c00 legacy=0x7f53380b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533cc08640 1 -- 172.21.15.8:0/3444801927 >> v1:172.21.15.8:6789/0 conn(0x7f5338071820 legacy=0x7f533815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.641+0000 7f533cc08640 1 -- 172.21.15.8:0/3444801927 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53380b2f00 con 0x7f53381577d0 2023-10-30T12:45:31.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533cc08640 1 --2- 172.21.15.8:0/3444801927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f5338157c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96219c4a98ec1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f5336ffd640 1 -- 172.21.15.8:0/3444801927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300a59e0 con 0x7f53381577d0 2023-10-30T12:45:31.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f5336ffd640 1 -- 172.21.15.8:0/3444801927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53300a5b40 con 0x7f53381577d0 2023-10-30T12:45:31.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f5336ffd640 1 -- 172.21.15.8:0/3444801927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300ad440 con 0x7f53381577d0 2023-10-30T12:45:31.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 -- 172.21.15.8:0/3444801927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 msgr2=0x7f5338157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 --2- 172.21.15.8:0/3444801927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f5338157c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533001c920 comp rx=0 tx=0).stop 2023-10-30T12:45:31.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 -- 172.21.15.8:0/3444801927 shutdown_connections 2023-10-30T12:45:31.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 --2- 172.21.15.8:0/3444801927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f5338157c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 -- 172.21.15.8:0/3444801927 >> 172.21.15.8:0/3444801927 conn(0x7f53380cef60 msgr2=0x7f53380cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 -- 172.21.15.8:0/3444801927 shutdown_connections 2023-10-30T12:45:31.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.645+0000 7f533d409640 1 -- 172.21.15.8:0/3444801927 wait complete. 2023-10-30T12:45:31.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 Processor -- start 2023-10-30T12:45:31.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- start start 2023-10-30T12:45:31.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 0x7f53381f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 0x7f5338154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5338154650 con 0x7f53381577d0 2023-10-30T12:45:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5338154060 con 0x7f5338071c00 2023-10-30T12:45:31.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53380b24e0 con 0x7f5338071820 2023-10-30T12:45:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55988/0 (socket says 172.21.15.8:55988) 2023-10-30T12:45:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 -- 172.21.15.8:0/1320329852 learned_addr learned my addr 172.21.15.8:0/1320329852 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f5337fff640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 0x7f53381f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f53377fe640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 0x7f5338154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 -- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 msgr2=0x7f53381f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 0x7f53381f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 -- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 msgr2=0x7f5338154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 0x7f5338154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 -- 172.21.15.8:0/1320329852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330009a70 con 0x7f53381577d0 2023-10-30T12:45:31.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f5337fff640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 0x7f53381f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f53377fe640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 0x7f5338154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533cc08640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f533001a550 tx=0x7f53300196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300addc0 con 0x7f53381577d0 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53300b9450 con 0x7f53381577d0 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300b97e0 con 0x7f53381577d0 2023-10-30T12:45:31.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53381f5760 con 0x7f53381577d0 2023-10-30T12:45:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.649+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338155cd0 con 0x7f53381577d0 2023-10-30T12:45:31.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338154790 con 0x7f53381577d0 2023-10-30T12:45:31.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53300b9940 con 0x7f53381577d0 2023-10-30T12:45:31.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f5334ff9640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 0x7f5324068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f533011f640 con 0x7f53381577d0 2023-10-30T12:45:31.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f5337fff640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 0x7f5324068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.653+0000 7f5337fff640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 0x7f5324068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5328000f70 tx=0x7f5328007c60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.657+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5338154790 con 0x7f53381577d0 2023-10-30T12:45:31.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.813+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53381f5760 con 0x7f53381577d0 2023-10-30T12:45:31.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.813+0000 7f5334ff9640 1 -- 172.21.15.8:0/1320329852 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53381f5760 con 0x7f53381577d0 2023-10-30T12:45:31.817 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:45:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 msgr2=0x7f5324068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 0x7f5324068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5328000f70 tx=0x7f5328007c60 comp rx=0 tx=0).stop 2023-10-30T12:45:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 msgr2=0x7f533814e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f533001a550 tx=0x7f53300196c0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 shutdown_connections 2023-10-30T12:45:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53240664a0 0x7f5324068960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5338071c00 0x7f5338154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53381577d0 0x7f533814e6a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 --2- 172.21.15.8:0/1320329852 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5338071820 0x7f53381f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 >> 172.21.15.8:0/1320329852 conn(0x7f53380cef60 msgr2=0x7f53380ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 shutdown_connections 2023-10-30T12:45:31.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.817+0000 7f533d409640 1 -- 172.21.15.8:0/1320329852 wait complete. 2023-10-30T12:45:31.838 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:45:31.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 Processor -- start 2023-10-30T12:45:31.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 -- start start 2023-10-30T12:45:31.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd58153ba0 0x7fcd580cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd580c4410 0x7fcd5815a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd580c4810 0x7fcd5815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58153f80 con 0x7fcd580c4810 2023-10-30T12:45:31.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58159df0 con 0x7fcd580c4410 2023-10-30T12:45:31.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd5d3a1640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5815a7c0 con 0x7fcd58153ba0 2023-10-30T12:45:31.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd580c4810 0x7fcd5815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd580c4810 0x7fcd5815cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55998/0 (socket says 172.21.15.8:55998) 2023-10-30T12:45:31.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 -- 172.21.15.8:0/2564497887 learned_addr learned my addr 172.21.15.8:0/2564497887 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:31.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd56ffd640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd58153ba0 0x7fcd580cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd4ffff640 1 --2- 172.21.15.8:0/2564497887 >> v2:172.21.15.46:3300/0 conn(0x7fcd580c4410 0x7fcd5815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 -- 172.21.15.8:0/2564497887 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd58153ba0 msgr2=0x7fcd580cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:31.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd58153ba0 0x7fcd580cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 -- 172.21.15.8:0/2564497887 >> v2:172.21.15.46:3300/0 conn(0x7fcd580c4410 msgr2=0x7fcd5815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:31.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 --2- 172.21.15.8:0/2564497887 >> v2:172.21.15.46:3300/0 conn(0x7fcd580c4410 0x7fcd5815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.981+0000 7fcd577fe640 1 -- 172.21.15.8:0/2564497887 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5815d2f0 con 0x7fcd580c4810 2023-10-30T12:45:31.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd577fe640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4810 0x7fcd5815cdb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806d180 tx=0x7fcd5001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90267f40de416b32 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd567fc640 1 -- 172.21.15.8:0/2564497887 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500a57c0 con 0x7fcd580c4810 2023-10-30T12:45:31.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd567fc640 1 -- 172.21.15.8:0/2564497887 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500a5920 con 0x7fcd580c4810 2023-10-30T12:45:31.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd567fc640 1 -- 172.21.15.8:0/2564497887 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500a5cb0 con 0x7fcd580c4810 2023-10-30T12:45:31.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/2564497887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4810 msgr2=0x7fcd5815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4810 0x7fcd5815cdb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806d180 tx=0x7fcd5001c940 comp rx=0 tx=0).stop 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/2564497887 shutdown_connections 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4810 0x7fcd5815cdb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/2564497887 >> v2:172.21.15.46:3300/0 conn(0x7fcd580c4410 0x7fcd5815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/2564497887 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd58153ba0 0x7fcd580cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/2564497887 >> 172.21.15.8:0/2564497887 conn(0x7fcd58071790 msgr2=0x7fcd58071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:31.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/2564497887 shutdown_connections 2023-10-30T12:45:31.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.985+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/2564497887 wait complete. 2023-10-30T12:45:31.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 Processor -- start 2023-10-30T12:45:31.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- start start 2023-10-30T12:45:31.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 0x7fcd5815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd58153ba0 0x7fcd5815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:31.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581f6060 con 0x7fcd580c4410 2023-10-30T12:45:31.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581f57c0 con 0x7fcd58153ba0 2023-10-30T12:45:31.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581f5aa0 con 0x7fcd580c4810 2023-10-30T12:45:31.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56014/0 (socket says 172.21.15.8:56014) 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 -- 172.21.15.8:0/1377578506 learned_addr learned my addr 172.21.15.8:0/1377578506 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd4ffff640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 0x7fcd5815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd577fe640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd58153ba0 0x7fcd5815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 -- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 msgr2=0x7fcd5815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 0x7fcd5815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 -- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd58153ba0 msgr2=0x7fcd5815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:31.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd58153ba0 0x7fcd5815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:31.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 -- 172.21.15.8:0/1377578506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009b30 con 0x7fcd580c4410 2023-10-30T12:45:31.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd4ffff640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 0x7fcd5815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:31.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd56ffd640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d5f0 tx=0x7fcd4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:31.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480125f0 con 0x7fcd580c4410 2023-10-30T12:45:31.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48012750 con 0x7fcd580c4410 2023-10-30T12:45:31.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd581f5d80 con 0x7fcd580c4410 2023-10-30T12:45:31.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48012ae0 con 0x7fcd580c4410 2023-10-30T12:45:31.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.989+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5815ba90 con 0x7fcd580c4410 2023-10-30T12:45:31.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5804e170 con 0x7fcd580c4410 2023-10-30T12:45:31.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd48012c40 con 0x7fcd580c4410 2023-10-30T12:45:31.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd4f7fe640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 0x7fcd38068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd480843f0 con 0x7fcd580c4410 2023-10-30T12:45:32.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd4ffff640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 0x7fcd38068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.993+0000 7fcd4ffff640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 0x7fcd38068990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd40004660 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:31.997+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd480517d0 con 0x7fcd580c4410 2023-10-30T12:45:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.153+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd580cabb0 con 0x7fcd580c4410 2023-10-30T12:45:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd4f7fe640 1 -- 172.21.15.8:0/1377578506 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd48007380 con 0x7fcd580c4410 2023-10-30T12:45:32.159 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:45:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 msgr2=0x7fcd38068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 0x7fcd38068990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd40004660 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2023-10-30T12:45:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 msgr2=0x7fcd581f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d5f0 tx=0x7fcd4800dab0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 shutdown_connections 2023-10-30T12:45:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcd380664d0 0x7fcd38068990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd58153ba0 0x7fcd5815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd580c4810 0x7fcd5815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 --2- 172.21.15.8:0/1377578506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd580c4410 0x7fcd581f5180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 >> 172.21.15.8:0/1377578506 conn(0x7fcd58071790 msgr2=0x7fcd58158680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 shutdown_connections 2023-10-30T12:45:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.157+0000 7fcd5d3a1640 1 -- 172.21.15.8:0/1377578506 wait complete. 2023-10-30T12:45:32.179 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:32.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 Processor -- start 2023-10-30T12:45:32.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 -- start start 2023-10-30T12:45:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c153c80 0x7f082c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f082c0c42a0 0x7f082c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c159f20 con 0x7f082c0c4a10 2023-10-30T12:45:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c15dec0 con 0x7f082c0c42a0 2023-10-30T12:45:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0833d3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1599a0 con 0x7f082c153c80 2023-10-30T12:45:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f082c0c42a0 0x7f082c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f082c0c42a0 0x7f082c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56016/0 (socket says 172.21.15.8:56016) 2023-10-30T12:45:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 -- 172.21.15.8:0/1736534056 learned_addr learned my addr 172.21.15.8:0/1736534056 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f0831ada640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c153c80 0x7f082c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 -- 172.21.15.8:0/1736534056 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c153c80 msgr2=0x7f082c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:32.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c153c80 0x7f082c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 -- 172.21.15.8:0/1736534056 >> v1:172.21.15.46:6789/0 conn(0x7f082c0c4a10 legacy=0x7f082c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.321+0000 7f08322db640 1 -- 172.21.15.8:0/1736534056 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c15d620 con 0x7f082c0c42a0 2023-10-30T12:45:32.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f08322db640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c0c42a0 0x7f082c0ca1e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0824009e50 tx=0x7f082401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599204c19ca91c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0830ad8640 1 -- 172.21.15.8:0/1736534056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240af400 con 0x7f082c0c42a0 2023-10-30T12:45:32.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0830ad8640 1 -- 172.21.15.8:0/1736534056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08240ab050 con 0x7f082c0c42a0 2023-10-30T12:45:32.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0830ad8640 1 -- 172.21.15.8:0/1736534056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240af780 con 0x7f082c0c42a0 2023-10-30T12:45:32.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 -- 172.21.15.8:0/1736534056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c0c42a0 msgr2=0x7f082c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c0c42a0 0x7f082c0ca1e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0824009e50 tx=0x7f082401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 -- 172.21.15.8:0/1736534056 shutdown_connections 2023-10-30T12:45:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c0c42a0 0x7f082c0ca1e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 --2- 172.21.15.8:0/1736534056 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c153c80 0x7f082c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 -- 172.21.15.8:0/1736534056 >> 172.21.15.8:0/1736534056 conn(0x7f082c0715c0 msgr2=0x7f082c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 -- 172.21.15.8:0/1736534056 shutdown_connections 2023-10-30T12:45:32.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.325+0000 7f0833d3e640 1 -- 172.21.15.8:0/1736534056 wait complete. 2023-10-30T12:45:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 Processor -- start 2023-10-30T12:45:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- start start 2023-10-30T12:45:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 0x7f082c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 0x7f082c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c14f310 con 0x7f082c153c80 2023-10-30T12:45:32.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c14fb50 con 0x7f082c0c42a0 2023-10-30T12:45:32.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c14e9e0 con 0x7f082c0c4a10 2023-10-30T12:45:32.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56022/0 (socket says 172.21.15.8:56022) 2023-10-30T12:45:32.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 -- 172.21.15.8:0/2809023216 learned_addr learned my addr 172.21.15.8:0/2809023216 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:32.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0831ada640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 0x7f082c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08312d9640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 0x7f082c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 -- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 msgr2=0x7f082c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 0x7f082c0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 -- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 msgr2=0x7f082c15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 0x7f082c15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 -- 172.21.15.8:0/2809023216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824009b30 con 0x7f082c153c80 2023-10-30T12:45:32.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0831ada640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 0x7f082c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:32.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08312d9640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 0x7f082c0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08322db640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f082401ce20 tx=0x7f082401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240afde0 con 0x7f082c153c80 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0824009510 con 0x7f082c153c80 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240ba660 con 0x7f082c153c80 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c14ecc0 con 0x7f082c153c80 2023-10-30T12:45:32.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c14fdf0 con 0x7f082c153c80 2023-10-30T12:45:32.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.329+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c14f4c0 con 0x7f082c153c80 2023-10-30T12:45:32.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.333+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08240b6040 con 0x7f082c153c80 2023-10-30T12:45:32.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.333+0000 7f08227fc640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 0x7f0810068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.333+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08241221f0 con 0x7f082c153c80 2023-10-30T12:45:32.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.333+0000 7f0831ada640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 0x7f0810068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.333+0000 7f0831ada640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 0x7f0810068960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f082800b9f0 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.337+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f082c14f4c0 con 0x7f082c153c80 2023-10-30T12:45:32.398 INFO:tasks.ceph.mds.a.smithi008.stderr:2023-10-30T12:45:32.397+0000 7fe0693b9640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:45:32.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.525+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f082c14f4c0 con 0x7f082c153c80 2023-10-30T12:45:32.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.525+0000 7f08227fc640 1 -- 172.21.15.8:0/2809023216 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+6509 (secure 0 0 0) 0x7f082c14f4c0 con 0x7f082c153c80 2023-10-30T12:45:32.530 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:32.530 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:30.181831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:32.531 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 11 2023-10-30T12:45:32.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 msgr2=0x7f0810068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 0x7f0810068960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f082800b9f0 tx=0x7f0828008040 comp rx=0 tx=0).stop 2023-10-30T12:45:32.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 msgr2=0x7f082c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f082401ce20 tx=0x7f082401bf90 comp rx=0 tx=0).stop 2023-10-30T12:45:32.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 shutdown_connections 2023-10-30T12:45:32.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08100664a0 0x7f0810068960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f082c153c80 0x7f082c0cd610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f082c0c4a10 0x7f082c0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 --2- 172.21.15.8:0/2809023216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0c42a0 0x7f082c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 >> 172.21.15.8:0/2809023216 conn(0x7f082c0715c0 msgr2=0x7f082c158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:32.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 shutdown_connections 2023-10-30T12:45:32.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.529+0000 7f0833d3e640 1 -- 172.21.15.8:0/2809023216 wait complete. 2023-10-30T12:45:32.551 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:32.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 Processor -- start 2023-10-30T12:45:32.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 -- start start 2023-10-30T12:45:32.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7204153c80 0x7f720415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72040c4a10 0x7f72040c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7204159f20 con 0x7f720415a3b0 2023-10-30T12:45:32.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720415dec0 con 0x7f7204153c80 2023-10-30T12:45:32.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f720aa32640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72041599a0 con 0x7f72040c4a10 2023-10-30T12:45:32.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7204153c80 0x7f720415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7204153c80 0x7f720415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56036/0 (socket says 172.21.15.8:56036) 2023-10-30T12:45:32.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7208fcf640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f720415a3b0 0x7f72040ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53920/0 (socket says 172.21.15.8:53920) 2023-10-30T12:45:32.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 -- 172.21.15.8:0/4094712927 learned_addr learned my addr 172.21.15.8:0/4094712927 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:32.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f72037fe640 1 --2- 172.21.15.8:0/4094712927 >> v2:172.21.15.46:3300/0 conn(0x7f72040c4a10 0x7f72040c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 -- 172.21.15.8:0/4094712927 >> v2:172.21.15.46:3300/0 conn(0x7f72040c4a10 msgr2=0x7f72040c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 --2- 172.21.15.8:0/4094712927 >> v2:172.21.15.46:3300/0 conn(0x7f72040c4a10 0x7f72040c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 -- 172.21.15.8:0/4094712927 >> v1:172.21.15.8:6789/0 conn(0x7f720415a3b0 legacy=0x7f72040ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.693+0000 7f7203fff640 1 -- 172.21.15.8:0/4094712927 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040c47e0 con 0x7f7204153c80 2023-10-30T12:45:32.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f7203fff640 1 --2- 172.21.15.8:0/4094712927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7204153c80 0x7f720415cf90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f720406be50 tx=0x7f71f401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5b9e15fe61a62f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f7202ffd640 1 -- 172.21.15.8:0/4094712927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40a89f0 con 0x7f7204153c80 2023-10-30T12:45:32.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f7202ffd640 1 -- 172.21.15.8:0/4094712927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f40a8b50 con 0x7f7204153c80 2023-10-30T12:45:32.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f7202ffd640 1 -- 172.21.15.8:0/4094712927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40b0440 con 0x7f7204153c80 2023-10-30T12:45:32.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 -- 172.21.15.8:0/4094712927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7204153c80 msgr2=0x7f720415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 --2- 172.21.15.8:0/4094712927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7204153c80 0x7f720415cf90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f720406be50 tx=0x7f71f401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 -- 172.21.15.8:0/4094712927 shutdown_connections 2023-10-30T12:45:32.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 --2- 172.21.15.8:0/4094712927 >> v2:172.21.15.46:3300/0 conn(0x7f72040c4a10 0x7f72040c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 --2- 172.21.15.8:0/4094712927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7204153c80 0x7f720415cf90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 -- 172.21.15.8:0/4094712927 >> 172.21.15.8:0/4094712927 conn(0x7f72040715c0 msgr2=0x7f72040719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:32.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 -- 172.21.15.8:0/4094712927 shutdown_connections 2023-10-30T12:45:32.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.697+0000 7f720aa32640 1 -- 172.21.15.8:0/4094712927 wait complete. 2023-10-30T12:45:32.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 Processor -- start 2023-10-30T12:45:32.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- start start 2023-10-30T12:45:32.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7204153c80 0x7f72040c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f720415a3b0 0x7f720414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b1be0 con 0x7f72040c4a10 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204146220 con 0x7f7204153c80 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7204152790 con 0x7f720415a3b0 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56050/0 (socket says 172.21.15.8:56050) 2023-10-30T12:45:32.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 -- 172.21.15.8:0/3121455322 learned_addr learned my addr 172.21.15.8:0/3121455322 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:32.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f72037fe640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7204153c80 0x7f72040c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7208fcf640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f720415a3b0 0x7f720414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 -- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f720415a3b0 msgr2=0x7f720414aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f720415a3b0 0x7f720414aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 -- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7204153c80 msgr2=0x7f72040c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7204153c80 0x7f72040c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 -- 172.21.15.8:0/3121455322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40094d0 con 0x7f72040c4a10 2023-10-30T12:45:32.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7203fff640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71f40b3040 tx=0x7f71f401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40b0e00 con 0x7f72040c4a10 2023-10-30T12:45:32.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f40bb470 con 0x7f72040c4a10 2023-10-30T12:45:32.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40bb820 con 0x7f72040c4a10 2023-10-30T12:45:32.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7204146490 con 0x7f72040c4a10 2023-10-30T12:45:32.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040c9480 con 0x7f72040c4a10 2023-10-30T12:45:32.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.701+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720404e170 con 0x7f72040c4a10 2023-10-30T12:45:32.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.705+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f71f40b7070 con 0x7f72040c4a10 2023-10-30T12:45:32.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.705+0000 7f7200ff9640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 0x7f71f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:32.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.705+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f41222f0 con 0x7f72040c4a10 2023-10-30T12:45:32.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.705+0000 7f72037fe640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 0x7f71f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:32.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.705+0000 7f72037fe640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 0x7f71f0068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f720415d4a0 tx=0x7f71f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:32.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.709+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71f40ef640 con 0x7f72040c4a10 2023-10-30T12:45:32.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.865+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72040cad40 con 0x7f72040c4a10 2023-10-30T12:45:32.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.865+0000 7f7200ff9640 1 -- 172.21.15.8:0/3121455322 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f71f40b5090 con 0x7f72040c4a10 2023-10-30T12:45:32.870 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:32.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 msgr2=0x7f71f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 0x7f71f0068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f720415d4a0 tx=0x7f71f8008040 comp rx=0 tx=0).stop 2023-10-30T12:45:32.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 msgr2=0x7f72040c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:32.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71f40b3040 tx=0x7f71f401ba90 comp rx=0 tx=0).stop 2023-10-30T12:45:32.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 shutdown_connections 2023-10-30T12:45:32.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f71f00664a0 0x7f71f0068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7204153c80 0x7f72040c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f720415a3b0 0x7f720414aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 --2- 172.21.15.8:0/3121455322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72040c4a10 0x7f72040c8670 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:32.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 >> 172.21.15.8:0/3121455322 conn(0x7f72040715c0 msgr2=0x7f72040caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:32.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 shutdown_connections 2023-10-30T12:45:32.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:32.869+0000 7f720aa32640 1 -- 172.21.15.8:0/3121455322 wait complete. 2023-10-30T12:45:32.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:32.892 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:33.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 Processor -- start 2023-10-30T12:45:33.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 -- start start 2023-10-30T12:45:33.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fef0c153ba0 0x7fef0c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef0c0c4410 0x7fef0c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c153f80 con 0x7fef0c0c4810 2023-10-30T12:45:33.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c159df0 con 0x7fef0c153ba0 2023-10-30T12:45:33.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef10a9c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c15a7c0 con 0x7fef0c0c4410 2023-10-30T12:45:33.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fef0c153ba0 0x7fef0c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fef0c153ba0 0x7fef0c0cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56058/0 (socket says 172.21.15.8:56058) 2023-10-30T12:45:33.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 -- 172.21.15.8:0/4256726903 learned_addr learned my addr 172.21.15.8:0/4256726903 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:33.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef03fff640 1 --2- 172.21.15.8:0/4256726903 >> v2:172.21.15.46:3300/0 conn(0x7fef0c0c4410 0x7fef0c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 -- 172.21.15.8:0/4256726903 >> v2:172.21.15.46:3300/0 conn(0x7fef0c0c4410 msgr2=0x7fef0c15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:33.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 --2- 172.21.15.8:0/4256726903 >> v2:172.21.15.46:3300/0 conn(0x7fef0c0c4410 0x7fef0c15a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 -- 172.21.15.8:0/4256726903 >> v1:172.21.15.46:6789/0 conn(0x7fef0c0c4810 legacy=0x7fef0c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.037+0000 7fef0a59c640 1 -- 172.21.15.8:0/4256726903 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c1597c0 con 0x7fef0c153ba0 2023-10-30T12:45:33.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef0a59c640 1 --2- 172.21.15.8:0/4256726903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c153ba0 0x7fef0c0cb4e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef0c072540 tx=0x7feefc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96d211a517fa116 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef09d9b640 1 -- 172.21.15.8:0/4256726903 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0ab640 con 0x7fef0c153ba0 2023-10-30T12:45:33.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef09d9b640 1 -- 172.21.15.8:0/4256726903 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0a7070 con 0x7fef0c153ba0 2023-10-30T12:45:33.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef09d9b640 1 -- 172.21.15.8:0/4256726903 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0ab9c0 con 0x7fef0c153ba0 2023-10-30T12:45:33.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 -- 172.21.15.8:0/4256726903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c153ba0 msgr2=0x7fef0c0cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 --2- 172.21.15.8:0/4256726903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c153ba0 0x7fef0c0cb4e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fef0c072540 tx=0x7feefc01c440 comp rx=0 tx=0).stop 2023-10-30T12:45:33.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 -- 172.21.15.8:0/4256726903 shutdown_connections 2023-10-30T12:45:33.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 --2- 172.21.15.8:0/4256726903 >> v2:172.21.15.46:3300/0 conn(0x7fef0c0c4410 0x7fef0c15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 --2- 172.21.15.8:0/4256726903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c153ba0 0x7fef0c0cb4e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 -- 172.21.15.8:0/4256726903 >> 172.21.15.8:0/4256726903 conn(0x7fef0c071790 msgr2=0x7fef0c071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 -- 172.21.15.8:0/4256726903 shutdown_connections 2023-10-30T12:45:33.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.041+0000 7fef10a9c640 1 -- 172.21.15.8:0/4256726903 wait complete. 2023-10-30T12:45:33.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 Processor -- start 2023-10-30T12:45:33.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- start start 2023-10-30T12:45:33.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0c4410 0x7fef0c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 0x7fef0c1f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c151ae0 con 0x7fef0c0c4810 2023-10-30T12:45:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c147e20 con 0x7fef0c0c4410 2023-10-30T12:45:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c1f2050 con 0x7fef0c153ba0 2023-10-30T12:45:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56074/0 (socket says 172.21.15.8:56074) 2023-10-30T12:45:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 -- 172.21.15.8:0/3754539503 learned_addr learned my addr 172.21.15.8:0/3754539503 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef0a59c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0c4410 0x7fef0c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef0ad9d640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 0x7fef0c1f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 -- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 msgr2=0x7fef0c1f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 0x7fef0c1f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 -- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0c4410 msgr2=0x7fef0c1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0c4410 0x7fef0c1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 -- 172.21.15.8:0/3754539503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0094d0 con 0x7fef0c0c4810 2023-10-30T12:45:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef0ad9d640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 0x7fef0c1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef03fff640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feef400d800 tx=0x7feef400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400b6c0 con 0x7fef0c0c4810 2023-10-30T12:45:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400b820 con 0x7fef0c0c4810 2023-10-30T12:45:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400bb90 con 0x7fef0c0c4810 2023-10-30T12:45:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c0b4640 con 0x7fef0c0c4810 2023-10-30T12:45:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.045+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c0b4c00 con 0x7fef0c0c4810 2023-10-30T12:45:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c04e170 con 0x7fef0c0c4810 2023-10-30T12:45:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feef400bcf0 con 0x7fef0c0c4810 2023-10-30T12:45:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef02ffd640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 0x7feeec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feef4084340 con 0x7fef0c0c4810 2023-10-30T12:45:33.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef0a59c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 0x7feeec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.049+0000 7fef0a59c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 0x7feeec068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc0b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.053+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feef40517b0 con 0x7fef0c0c4810 2023-10-30T12:45:33.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef0c0cabb0 con 0x7fef0c0c4810 2023-10-30T12:45:33.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef02ffd640 1 -- 172.21.15.8:0/3754539503 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7feef4054dd0 con 0x7fef0c0c4810 2023-10-30T12:45:33.216 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:33.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 msgr2=0x7feeec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 0x7feeec068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc0b6040 comp rx=0 tx=0).stop 2023-10-30T12:45:33.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 msgr2=0x7fef0c0c5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feef400d800 tx=0x7feef400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 shutdown_connections 2023-10-30T12:45:33.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feeec0664a0 0x7feeec068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fef0c153ba0 0x7fef0c1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef0c0c4810 0x7fef0c0c5010 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 --2- 172.21.15.8:0/3754539503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0c4410 0x7fef0c1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 >> 172.21.15.8:0/3754539503 conn(0x7fef0c071790 msgr2=0x7fef0c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 shutdown_connections 2023-10-30T12:45:33.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.213+0000 7fef10a9c640 1 -- 172.21.15.8:0/3754539503 wait complete. 2023-10-30T12:45:33.237 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:45:33.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 Processor -- start 2023-10-30T12:45:33.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 -- start start 2023-10-30T12:45:33.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f957c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c153f80 con 0x7f957c153ba0 2023-10-30T12:45:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c159df0 con 0x7f957c0c4410 2023-10-30T12:45:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f958413e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1597c0 con 0x7f957c0c4810 2023-10-30T12:45:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9581eda640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f957c153ba0 0x7f957c15cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53928/0 (socket says 172.21.15.8:53928) 2023-10-30T12:45:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9581eda640 1 -- 172.21.15.8:0/1929421653 learned_addr learned my addr 172.21.15.8:0/1929421653 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f957c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296422332 0 0) 0x7f957c153f80 con 0x7f957c153ba0 2023-10-30T12:45:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003530 con 0x7f957c153ba0 2023-10-30T12:45:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 713511894 0 0) 0x7f9568003530 con 0x7f957c153ba0 2023-10-30T12:45:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 msgr2=0x7f957c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f957c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 >> v1:172.21.15.46:6789/0 conn(0x7f957c0c4410 legacy=0x7f957c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c15d580 con 0x7f957c153ba0 2023-10-30T12:45:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f9578001680 con 0x7f957c153ba0 2023-10-30T12:45:33.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 >> v1:172.21.15.8:6789/0 conn(0x7f957c153ba0 legacy=0x7f957c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95680039b0 0x7f9568003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f95680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9568004d10 0x7f9568005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1597c0 con 0x7f9568004d10 2023-10-30T12:45:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956800ba40 con 0x7f95680039b0 2023-10-30T12:45:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956800bc00 con 0x7f957c0c4810 2023-10-30T12:45:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9568004d10 0x7f9568005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9581eda640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95680039b0 0x7f9568003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95816d9640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f95680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 -- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 msgr2=0x7f95680047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c0c4810 0x7f95680047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 -- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95680039b0 msgr2=0x7f9568003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95680039b0 0x7f9568003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f95826db640 1 -- 172.21.15.8:0/1929421653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568003530 con 0x7f9568004d10 2023-10-30T12:45:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.381+0000 7f9581eda640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95680039b0 0x7f9568003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f95826db640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9568004d10 0x7f9568005190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9574009f90 tx=0x7f957401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1023df4805a0b505 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740a5af0 con 0x7f9568004d10 2023-10-30T12:45:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957400f040 con 0x7f9568004d10 2023-10-30T12:45:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f9580ed8640 1 -- 172.21.15.8:0/1929421653 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740a5e50 con 0x7f9568004d10 2023-10-30T12:45:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 -- 172.21.15.8:0/1929421653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9568004d10 msgr2=0x7f9568005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 --2- 172.21.15.8:0/1929421653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9568004d10 0x7f9568005190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9574009f90 tx=0x7f957401cbf0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f95826db640 1 -- 172.21.15.8:0/1929421653 reap_dead start 2023-10-30T12:45:33.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 -- 172.21.15.8:0/1929421653 shutdown_connections 2023-10-30T12:45:33.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 -- 172.21.15.8:0/1929421653 >> 172.21.15.8:0/1929421653 conn(0x7f957c071790 msgr2=0x7f957c071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 -- 172.21.15.8:0/1929421653 shutdown_connections 2023-10-30T12:45:33.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.385+0000 7f958413e640 1 -- 172.21.15.8:0/1929421653 wait complete. 2023-10-30T12:45:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 Processor -- start 2023-10-30T12:45:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- start start 2023-10-30T12:45:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 0x7f957c0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c158020 0x7f957c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0b35c0 con 0x7f957c151f90 2023-10-30T12:45:33.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c15c2c0 con 0x7f957c157c40 2023-10-30T12:45:33.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1524f0 con 0x7f957c158020 2023-10-30T12:45:33.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56090/0 (socket says 172.21.15.8:56090) 2023-10-30T12:45:33.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 -- 172.21.15.8:0/497156777 learned_addr learned my addr 172.21.15.8:0/497156777 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:33.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f9581eda640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 0x7f957c0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95816d9640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c158020 0x7f957c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 -- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c158020 msgr2=0x7f957c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c158020 0x7f957c15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 -- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 msgr2=0x7f957c0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 0x7f957c0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 -- 172.21.15.8:0/497156777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c15a7f0 con 0x7f957c151f90 2023-10-30T12:45:33.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f9581eda640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 0x7f957c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:33.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95826db640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9574009f90 tx=0x7f9574009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740ad640 con 0x7f957c151f90 2023-10-30T12:45:33.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95740ad7c0 con 0x7f957c151f90 2023-10-30T12:45:33.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c15b6b0 con 0x7f957c151f90 2023-10-30T12:45:33.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740adb70 con 0x7f957c151f90 2023-10-30T12:45:33.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.389+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c0b4b00 con 0x7f957c151f90 2023-10-30T12:45:33.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c0b3700 con 0x7f957c151f90 2023-10-30T12:45:33.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f95740b4040 con 0x7f957c151f90 2023-10-30T12:45:33.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f95727fc640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 0x7f9560068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f957411e320 con 0x7f957c151f90 2023-10-30T12:45:33.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f9581eda640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 0x7f9560068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.393+0000 7f9581eda640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 0x7f9560068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9578004850 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.397+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f957c0b3700 con 0x7f957c151f90 2023-10-30T12:45:33.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.553+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f957c0b3700 con 0x7f957c151f90 2023-10-30T12:45:33.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.553+0000 7f95727fc640 1 -- 172.21.15.8:0/497156777 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f957c0b3700 con 0x7f957c151f90 2023-10-30T12:45:33.558 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:45:33.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 msgr2=0x7f9560068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 0x7f9560068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9578004850 tx=0x7f9578009040 comp rx=0 tx=0).stop 2023-10-30T12:45:33.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 msgr2=0x7f957c15b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9574009f90 tx=0x7f9574009750 comp rx=0 tx=0).stop 2023-10-30T12:45:33.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 shutdown_connections 2023-10-30T12:45:33.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95600664a0 0x7f9560068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f957c151f90 0x7f957c15b170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f957c158020 0x7f957c15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 --2- 172.21.15.8:0/497156777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c157c40 0x7f957c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 >> 172.21.15.8:0/497156777 conn(0x7f957c071790 msgr2=0x7f957c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 shutdown_connections 2023-10-30T12:45:33.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.557+0000 7f958413e640 1 -- 172.21.15.8:0/497156777 wait complete. 2023-10-30T12:45:33.579 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:45:33.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.721+0000 7f43de69a640 1 Processor -- start 2023-10-30T12:45:33.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.721+0000 7f43de69a640 1 -- start start 2023-10-30T12:45:33.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43de69a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f43d80c4810 0x7f43d80cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43de69a640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8153f80 con 0x7f43d8153ba0 2023-10-30T12:45:33.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43de69a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8159df0 con 0x7f43d80c4410 2023-10-30T12:45:33.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43de69a640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d81597c0 con 0x7f43d80c4810 2023-10-30T12:45:33.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43d7fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f43d8153ba0 0x7f43d815cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53944/0 (socket says 172.21.15.8:53944) 2023-10-30T12:45:33.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43d7fff640 1 -- 172.21.15.8:0/1440851256 learned_addr learned my addr 172.21.15.8:0/1440851256 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:33.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43dcc37640 1 --2- 172.21.15.8:0/1440851256 >> v2:172.21.15.8:3300/0 conn(0x7f43d80c4810 0x7f43d80cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43dcc37640 1 -- 172.21.15.8:0/1440851256 >> v1:172.21.15.46:6789/0 conn(0x7f43d80c4410 legacy=0x7f43d80cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43dcc37640 1 -- 172.21.15.8:0/1440851256 >> v1:172.21.15.8:6789/0 conn(0x7f43d8153ba0 legacy=0x7f43d815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43dcc37640 1 -- 172.21.15.8:0/1440851256 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d815d580 con 0x7f43d80c4810 2023-10-30T12:45:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43dcc37640 1 --2- 172.21.15.8:0/1440851256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4810 0x7f43d80cb4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009d90 tx=0x7f43d001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb23d5e58ed3a5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.725+0000 7f43d6ffd640 1 -- 172.21.15.8:0/1440851256 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00a5950 con 0x7f43d80c4810 2023-10-30T12:45:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d6ffd640 1 -- 172.21.15.8:0/1440851256 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00a5ab0 con 0x7f43d80c4810 2023-10-30T12:45:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d6ffd640 1 -- 172.21.15.8:0/1440851256 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00a5e40 con 0x7f43d80c4810 2023-10-30T12:45:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- 172.21.15.8:0/1440851256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4810 msgr2=0x7f43d80cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 --2- 172.21.15.8:0/1440851256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4810 0x7f43d80cb4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009d90 tx=0x7f43d001c890 comp rx=0 tx=0).stop 2023-10-30T12:45:33.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- 172.21.15.8:0/1440851256 shutdown_connections 2023-10-30T12:45:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 --2- 172.21.15.8:0/1440851256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4810 0x7f43d80cb4e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- 172.21.15.8:0/1440851256 >> 172.21.15.8:0/1440851256 conn(0x7f43d8071790 msgr2=0x7f43d8071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- 172.21.15.8:0/1440851256 shutdown_connections 2023-10-30T12:45:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- 172.21.15.8:0/1440851256 wait complete. 2023-10-30T12:45:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 Processor -- start 2023-10-30T12:45:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- start start 2023-10-30T12:45:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f43d80c4810 0x7f43d81f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43d8153ba0 0x7f43d815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81f4db0 con 0x7f43d80c4410 2023-10-30T12:45:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8150ed0 con 0x7f43d8153ba0 2023-10-30T12:45:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43de69a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8151140 con 0x7f43d80c4810 2023-10-30T12:45:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56116/0 (socket says 172.21.15.8:56116) 2023-10-30T12:45:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d7fff640 1 -- 172.21.15.8:0/4228277373 learned_addr learned my addr 172.21.15.8:0/4228277373 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43dcc37640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43d8153ba0 0x7f43d815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.729+0000 7f43d77fe640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f43d80c4810 0x7f43d81f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 -- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f43d80c4810 msgr2=0x7f43d81f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f43d80c4810 0x7f43d81f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 -- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43d8153ba0 msgr2=0x7f43d815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43d8153ba0 0x7f43d815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 -- 172.21.15.8:0/4228277373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009a70 con 0x7f43d80c4410 2023-10-30T12:45:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d7fff640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8005cf0 con 0x7f43d80c4410 2023-10-30T12:45:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8005e50 con 0x7f43d80c4410 2023-10-30T12:45:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80072d0 con 0x7f43d80c4410 2023-10-30T12:45:33.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d815a7f0 con 0x7f43d80c4410 2023-10-30T12:45:33.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81f52e0 con 0x7f43d80c4410 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d804e170 con 0x7f43d80c4410 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f43c8007430 con 0x7f43d80c4410 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 0x7f43c40689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43c80833e0 con 0x7f43d80c4410 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.733+0000 7f43d77fe640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 0x7f43c40689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:33.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.737+0000 7f43d77fe640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 0x7f43c40689b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43d80720b0 tx=0x7f43cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:33.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.737+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f43c8050860 con 0x7f43d80c4410 2023-10-30T12:45:33.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43d80cabb0 con 0x7f43d80c4410 2023-10-30T12:45:33.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43d4ff9640 1 -- 172.21.15.8:0/4228277373 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43c8053e80 con 0x7f43d80c4410 2023-10-30T12:45:33.900 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:45:33.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 msgr2=0x7f43c40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 0x7f43c40689b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43d80720b0 tx=0x7f43cc009040 comp rx=0 tx=0).stop 2023-10-30T12:45:33.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 msgr2=0x7f43d8152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:33.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 shutdown_connections 2023-10-30T12:45:33.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f43c40664f0 0x7f43c40689b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43d8153ba0 0x7f43d815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f43d80c4810 0x7f43d81f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 --2- 172.21.15.8:0/4228277373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f43d80c4410 0x7f43d8152b20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:33.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.897+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 >> 172.21.15.8:0/4228277373 conn(0x7f43d8071790 msgr2=0x7f43d80c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:33.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.901+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 shutdown_connections 2023-10-30T12:45:33.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:33.901+0000 7f43de69a640 1 -- 172.21.15.8:0/4228277373 wait complete. 2023-10-30T12:45:33.921 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 Processor -- start 2023-10-30T12:45:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 -- start start 2023-10-30T12:45:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f16cc06bb40 0x7f16cc157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 0x7f16cc157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc071e00 con 0x7f16cc06b760 2023-10-30T12:45:34.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc070020 con 0x7f16cc06bb40 2023-10-30T12:45:34.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d3a32640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc070f10 con 0x7f16cc06c420 2023-10-30T12:45:34.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d0fcd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f16cc06bb40 0x7f16cc157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d0fcd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f16cc06bb40 0x7f16cc157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56122/0 (socket says 172.21.15.8:56122) 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d0fcd640 1 -- 172.21.15.8:0/1672542772 learned_addr learned my addr 172.21.15.8:0/1672542772 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.061+0000 7f16d1fcf640 1 --2- 172.21.15.8:0/1672542772 >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 0x7f16cc157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d0fcd640 1 -- 172.21.15.8:0/1672542772 >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 msgr2=0x7f16cc157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d0fcd640 1 --2- 172.21.15.8:0/1672542772 >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 0x7f16cc157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d0fcd640 1 -- 172.21.15.8:0/1672542772 >> v1:172.21.15.46:6789/0 conn(0x7f16cc06b760 legacy=0x7f16cc15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d0fcd640 1 -- 172.21.15.8:0/1672542772 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0711f0 con 0x7f16cc06bb40 2023-10-30T12:45:34.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d1fcf640 1 --2- 172.21.15.8:0/1672542772 >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 0x7f16cc157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:34.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d0fcd640 1 --2- 172.21.15.8:0/1672542772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06bb40 0x7f16cc157a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16cc15b7a0 tx=0x7f16bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce32974f7e336e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16c3fff640 1 -- 172.21.15.8:0/1672542772 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0aa430 con 0x7f16cc06bb40 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16c3fff640 1 -- 172.21.15.8:0/1672542772 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc0aa590 con 0x7f16cc06bb40 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16c3fff640 1 -- 172.21.15.8:0/1672542772 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0aa920 con 0x7f16cc06bb40 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 -- 172.21.15.8:0/1672542772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06bb40 msgr2=0x7f16cc157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1672542772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06bb40 0x7f16cc157a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16cc15b7a0 tx=0x7f16bc01c4d0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 -- 172.21.15.8:0/1672542772 shutdown_connections 2023-10-30T12:45:34.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1672542772 >> v2:172.21.15.46:3300/0 conn(0x7f16cc06c420 0x7f16cc157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1672542772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06bb40 0x7f16cc157a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.065+0000 7f16d3a32640 1 -- 172.21.15.8:0/1672542772 >> 172.21.15.8:0/1672542772 conn(0x7f16cc0cac70 msgr2=0x7f16cc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- 172.21.15.8:0/1672542772 shutdown_connections 2023-10-30T12:45:34.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- 172.21.15.8:0/1672542772 wait complete. 2023-10-30T12:45:34.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 Processor -- start 2023-10-30T12:45:34.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- start start 2023-10-30T12:45:34.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 0x7f16cc1f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16cc06bb40 0x7f16cc1f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1fa1b0 con 0x7f16cc06c420 2023-10-30T12:45:34.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1f9910 con 0x7f16cc06b760 2023-10-30T12:45:34.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1f9bf0 con 0x7f16cc06bb40 2023-10-30T12:45:34.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56132/0 (socket says 172.21.15.8:56132) 2023-10-30T12:45:34.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 -- 172.21.15.8:0/1311733169 learned_addr learned my addr 172.21.15.8:0/1311733169 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:34.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d17ce640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 0x7f16cc1f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d0fcd640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16cc06bb40 0x7f16cc1f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 -- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16cc06bb40 msgr2=0x7f16cc1f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16cc06bb40 0x7f16cc1f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 -- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 msgr2=0x7f16cc1f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 0x7f16cc1f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 -- 172.21.15.8:0/1311733169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0094d0 con 0x7f16cc06c420 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d17ce640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 0x7f16cc1f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d1fcf640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007df0 tx=0x7f16c400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c40039a0 con 0x7f16cc06c420 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c400f040 con 0x7f16cc06c420 2023-10-30T12:45:34.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4003d40 con 0x7f16cc06c420 2023-10-30T12:45:34.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.069+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc1f9ed0 con 0x7f16cc06c420 2023-10-30T12:45:34.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc1f6190 con 0x7f16cc06c420 2023-10-30T12:45:34.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc1f5d10 con 0x7f16cc06c420 2023-10-30T12:45:34.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f16c4004030 con 0x7f16cc06c420 2023-10-30T12:45:34.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16c1ffb640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 0x7f16b00689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16c4084270 con 0x7f16cc06c420 2023-10-30T12:45:34.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.073+0000 7f16d17ce640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 0x7f16b00689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.077+0000 7f16d17ce640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 0x7f16b00689b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000f70 tx=0x7f16c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.077+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f16cc1f5d10 con 0x7f16cc06c420 2023-10-30T12:45:34.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.269+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16cc1f5d10 con 0x7f16cc06c420 2023-10-30T12:45:34.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.269+0000 7f16c1ffb640 1 -- 172.21.15.8:0/1311733169 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+6509 (secure 0 0 0) 0x7f16cc1f5d10 con 0x7f16cc06c420 2023-10-30T12:45:34.274 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:34.274 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:30.181831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:34.275 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 11 2023-10-30T12:45:34.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 msgr2=0x7f16b00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 0x7f16b00689b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000f70 tx=0x7f16c8009040 comp rx=0 tx=0).stop 2023-10-30T12:45:34.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 msgr2=0x7f16cc157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007df0 tx=0x7f16c400d4a0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 shutdown_connections 2023-10-30T12:45:34.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16b00664f0 0x7f16b00689b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc06c420 0x7f16cc157a50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16cc06bb40 0x7f16cc1f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 --2- 172.21.15.8:0/1311733169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16cc06b760 0x7f16cc1f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 >> 172.21.15.8:0/1311733169 conn(0x7f16cc0cac70 msgr2=0x7f16cc153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 shutdown_connections 2023-10-30T12:45:34.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.273+0000 7f16d3a32640 1 -- 172.21.15.8:0/1311733169 wait complete. 2023-10-30T12:45:34.295 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:34.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 Processor -- start 2023-10-30T12:45:34.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 -- start start 2023-10-30T12:45:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb57c0c4a10 0x7fb57c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c159f20 con 0x7fb57c153c80 2023-10-30T12:45:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c15dec0 con 0x7fb57c0c4a10 2023-10-30T12:45:34.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb580e09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c1599a0 con 0x7fb57c0c42a0 2023-10-30T12:45:34.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb573fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb57c0c4a10 0x7fb57c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb57a59c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb57c153c80 0x7fb57c15a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45208/0 (socket says 172.21.15.8:45208) 2023-10-30T12:45:34.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb57ad9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb57a59c640 1 -- 172.21.15.8:0/474852042 learned_addr learned my addr 172.21.15.8:0/474852042 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:34.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb579d9b640 1 -- 172.21.15.8:0/474852042 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220287613 0 0) 0x7fb57c159f20 con 0x7fb57c153c80 2023-10-30T12:45:34.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.437+0000 7fb579d9b640 1 -- 172.21.15.8:0/474852042 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb568003530 con 0x7fb57c153c80 2023-10-30T12:45:34.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb573fff640 1 -- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 msgr2=0x7fb57c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb573fff640 1 --2- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb573fff640 1 -- 172.21.15.8:0/474852042 >> v1:172.21.15.46:6789/0 conn(0x7fb57c153c80 legacy=0x7fb57c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb573fff640 1 -- 172.21.15.8:0/474852042 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c15d620 con 0x7fb57c0c4a10 2023-10-30T12:45:34.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb573fff640 1 --2- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c15cf90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5640097c0 tx=0x7fb56401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc75b9c7581c2c62 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb579d9b640 1 -- 172.21.15.8:0/474852042 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640a8970 con 0x7fb57c0c4a10 2023-10-30T12:45:34.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb579d9b640 1 -- 172.21.15.8:0/474852042 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5640a8ad0 con 0x7fb57c0c4a10 2023-10-30T12:45:34.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.441+0000 7fb579d9b640 1 -- 172.21.15.8:0/474852042 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640a8e60 con 0x7fb57c0c4a10 2023-10-30T12:45:34.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 msgr2=0x7fb57c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c15cf90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5640097c0 tx=0x7fb56401ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:34.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- 172.21.15.8:0/474852042 shutdown_connections 2023-10-30T12:45:34.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- 172.21.15.8:0/474852042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c15cf90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- 172.21.15.8:0/474852042 >> 172.21.15.8:0/474852042 conn(0x7fb57c0715c0 msgr2=0x7fb57c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- 172.21.15.8:0/474852042 shutdown_connections 2023-10-30T12:45:34.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- 172.21.15.8:0/474852042 wait complete. 2023-10-30T12:45:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 Processor -- start 2023-10-30T12:45:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- start start 2023-10-30T12:45:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c14a4e0 con 0x7fb57c153c80 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c14bad0 con 0x7fb57c0c4a10 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb580e09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c14ef10 con 0x7fb57c0c42a0 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56146/0 (socket says 172.21.15.8:56146) 2023-10-30T12:45:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 -- 172.21.15.8:0/3461539514 learned_addr learned my addr 172.21.15.8:0/3461539514 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57a59c640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb573fff640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 -- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 msgr2=0x7fb57c14e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c14e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 -- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 msgr2=0x7fb57c149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 -- 172.21.15.8:0/3461539514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564009420 con 0x7fb57c153c80 2023-10-30T12:45:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb573fff640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.445+0000 7fb57ad9d640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb57400efc0 tx=0x7fb57400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574014480 con 0x7fb57c153c80 2023-10-30T12:45:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400f040 con 0x7fb57c153c80 2023-10-30T12:45:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574014820 con 0x7fb57c153c80 2023-10-30T12:45:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c14f1f0 con 0x7fb57c153c80 2023-10-30T12:45:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c14c130 con 0x7fb57c153c80 2023-10-30T12:45:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c04e170 con 0x7fb57c153c80 2023-10-30T12:45:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb574014980 con 0x7fb57c153c80 2023-10-30T12:45:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 0x7fb55c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb574084e70 con 0x7fb57c153c80 2023-10-30T12:45:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.449+0000 7fb57a59c640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 0x7fb55c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.453+0000 7fb57a59c640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 0x7fb55c068990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0011f0 tx=0x7fb56c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.453+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5740521c0 con 0x7fb57c153c80 2023-10-30T12:45:34.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb57c0b1b20 con 0x7fb57c153c80 2023-10-30T12:45:34.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb572ffd640 1 -- 172.21.15.8:0/3461539514 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7fb574018020 con 0x7fb57c153c80 2023-10-30T12:45:34.616 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:34.617 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:34.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 msgr2=0x7fb55c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 0x7fb55c068990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0011f0 tx=0x7fb56c009040 comp rx=0 tx=0).stop 2023-10-30T12:45:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 msgr2=0x7fb57c14cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb57400efc0 tx=0x7fb57400c490 comp rx=0 tx=0).stop 2023-10-30T12:45:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 shutdown_connections 2023-10-30T12:45:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb55c0664d0 0x7fb55c068990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb57c0c4a10 0x7fb57c149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb57c153c80 0x7fb57c14cc10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 --2- 172.21.15.8:0/3461539514 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb57c0c42a0 0x7fb57c14e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.613+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 >> 172.21.15.8:0/3461539514 conn(0x7fb57c0715c0 msgr2=0x7fb57c150e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.617+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 shutdown_connections 2023-10-30T12:45:34.620 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.617+0000 7fb580e09640 1 -- 172.21.15.8:0/3461539514 wait complete. 2023-10-30T12:45:34.638 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:34.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.777+0000 7f732349c640 1 Processor -- start 2023-10-30T12:45:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.777+0000 7f732349c640 1 -- start start 2023-10-30T12:45:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f732349c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f731c0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f732349c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f731c0cb8b0 0x7f731c06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f732349c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0c5030 con 0x7f731c151c20 2023-10-30T12:45:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f732349c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c157830 con 0x7f731c0cb8b0 2023-10-30T12:45:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f732349c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c157b10 con 0x7f731c151840 2023-10-30T12:45:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7320a37640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f731c151c20 0x7f731c072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53958/0 (socket says 172.21.15.8:53958) 2023-10-30T12:45:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7320a37640 1 -- 172.21.15.8:0/1380971138 learned_addr learned my addr 172.21.15.8:0/1380971138 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f731c0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321a39640 1 --2- 172.21.15.8:0/1380971138 >> v2:172.21.15.46:3300/0 conn(0x7f731c0cb8b0 0x7f731c06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233472189 0 0) 0x7f731c0c5030 con 0x7f731c151c20 2023-10-30T12:45:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7308003530 con 0x7f731c151c20 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228600080 0 0) 0x7f7308003530 con 0x7f731c151c20 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 msgr2=0x7f731c0c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f731c0c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 >> v2:172.21.15.46:3300/0 conn(0x7f731c0cb8b0 msgr2=0x7f731c06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 --2- 172.21.15.8:0/1380971138 >> v2:172.21.15.46:3300/0 conn(0x7f731c0cb8b0 0x7f731c06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c157df0 con 0x7f731c151c20 2023-10-30T12:45:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f730c001680 con 0x7f731c151c20 2023-10-30T12:45:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 >> v1:172.21.15.8:6789/0 conn(0x7f731c151c20 legacy=0x7f731c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7308003980 0x7f7308003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f73080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308004d10 0x7f7308005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c157830 con 0x7f7308003980 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c157b10 con 0x7f7308004d10 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730800b990 con 0x7f731c151840 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7308003980 0x7f7308003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7320a37640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f73080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321a39640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308004d10 0x7f7308005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 -- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 msgr2=0x7f73080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f731c151840 0x7f73080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 -- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308004d10 msgr2=0x7f7308005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308004d10 0x7f7308005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.781+0000 7f7321238640 1 -- 172.21.15.8:0/1380971138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308003530 con 0x7f7308003980 2023-10-30T12:45:34.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f7321238640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7308003980 0x7f7308003da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7318009ae0 tx=0x7f731801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f230d8710130dc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180a5530 con 0x7f7308003980 2023-10-30T12:45:34.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73180a5690 con 0x7f7308003980 2023-10-30T12:45:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f7313fff640 1 -- 172.21.15.8:0/1380971138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180a5a80 con 0x7f7308003980 2023-10-30T12:45:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 -- 172.21.15.8:0/1380971138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7308003980 msgr2=0x7f7308003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 --2- 172.21.15.8:0/1380971138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7308003980 0x7f7308003da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7318009ae0 tx=0x7f731801ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f7321a39640 1 -- 172.21.15.8:0/1380971138 reap_dead start 2023-10-30T12:45:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 -- 172.21.15.8:0/1380971138 shutdown_connections 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 -- 172.21.15.8:0/1380971138 >> 172.21.15.8:0/1380971138 conn(0x7f731c0cac70 msgr2=0x7f731c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 -- 172.21.15.8:0/1380971138 shutdown_connections 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.785+0000 7f732349c640 1 -- 172.21.15.8:0/1380971138 wait complete. 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 Processor -- start 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- start start 2023-10-30T12:45:34.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7308003980 0x7f731c1e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c1563b0 0x7f731c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c1e1860 con 0x7f731c14bfe0 2023-10-30T12:45:34.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c157390 con 0x7f731c1563b0 2023-10-30T12:45:34.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c155e70 con 0x7f7308003980 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56162/0 (socket says 172.21.15.8:56162) 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7321238640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7308003980 0x7f731c1e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 -- 172.21.15.8:0/2441473646 learned_addr learned my addr 172.21.15.8:0/2441473646 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7321a39640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c1563b0 0x7f731c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 -- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7308003980 msgr2=0x7f731c1e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7308003980 0x7f731c1e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 -- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c1563b0 msgr2=0x7f731c156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c1563b0 0x7f731c156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 -- 172.21.15.8:0/2441473646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c14bb50 con 0x7f731c14bfe0 2023-10-30T12:45:34.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7320a37640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f730c011950 tx=0x7f730c011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00fb00 con 0x7f731c14bfe0 2023-10-30T12:45:34.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c00fca0 con 0x7f731c14bfe0 2023-10-30T12:45:34.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c015650 con 0x7f731c14bfe0 2023-10-30T12:45:34.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c14c9a0 con 0x7f731c14bfe0 2023-10-30T12:45:34.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.789+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c1570b0 con 0x7f731c14bfe0 2023-10-30T12:45:34.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c14bce0 con 0x7f731c14bfe0 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f730c0157b0 con 0x7f731c14bfe0 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f7311ffb640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 0x7f7300068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f730c087290 con 0x7f731c14bfe0 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f7321238640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 0x7f7300068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.793+0000 7f7321238640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 0x7f7300068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73180af000 tx=0x7f73180046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:34.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.797+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f730c0545e0 con 0x7f731c14bfe0 2023-10-30T12:45:34.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f731c066510 con 0x7f731c14bfe0 2023-10-30T12:45:34.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f7311ffb640 1 -- 172.21.15.8:0/2441473646 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+6509 (secure 0 0 0) 0x7f730c057c00 con 0x7f731c14bfe0 2023-10-30T12:45:34.992 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:34.992 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:30.181831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 11 2023-10-30T12:45:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 msgr2=0x7f7300068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 0x7f7300068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f73180af000 tx=0x7f73180046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 msgr2=0x7f731c14c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f730c011950 tx=0x7f730c011e10 comp rx=0 tx=0).stop 2023-10-30T12:45:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 shutdown_connections 2023-10-30T12:45:34.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f73000664a0 0x7f7300068960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c1563b0 0x7f731c156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f731c14bfe0 0x7f731c14c460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 --2- 172.21.15.8:0/2441473646 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7308003980 0x7f731c1e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:34.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 >> 172.21.15.8:0/2441473646 conn(0x7f731c0cac70 msgr2=0x7f731c147c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:34.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 shutdown_connections 2023-10-30T12:45:34.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:34.989+0000 7f732349c640 1 -- 172.21.15.8:0/2441473646 wait complete. 2023-10-30T12:45:35.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-30T12:45:18.463017+0000', 'modified': '2023-10-30T12:45:30.181831+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4303, 'mds_1': 4276, 'mds_2': 4247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4247': {'gid': 4247, 'name': 'c', 'rank': 2, 'incarnation': 11, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.8:6839/710307090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6836', 'nonce': 710307090}, {'type': 'v1', 'addr': '172.21.15.8:6839', 'nonce': 710307090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4276': {'gid': 4276, 'name': 'f', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6833/3888673612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 3888673612}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 3888673612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4303': {'gid': 4303, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.8:6837/2926864154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6834', 'nonce': 2926864154}, {'type': 'v1', 'addr': '172.21.15.8:6837', 'nonce': 2926864154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-30T12:45:35.617 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4247:up:creating 2023-10-30T12:45:36.619 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:36.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 Processor -- start 2023-10-30T12:45:36.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 -- start start 2023-10-30T12:45:36.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 0x7ff1d415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d4159f20 con 0x7ff1d415a3b0 2023-10-30T12:45:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d415dec0 con 0x7ff1d40c4a10 2023-10-30T12:45:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.765+0000 7ff1dc565640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d41599a0 con 0x7ff1d4153c80 2023-10-30T12:45:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1d9b00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 0x7ff1d415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1d9b00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:45508/0 (socket says 172.21.15.8:45508) 2023-10-30T12:45:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1d9b00640 1 -- 172.21.15.8:0/2128949680 learned_addr learned my addr 172.21.15.8:0/2128949680 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 -- 172.21.15.8:0/2128949680 >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 msgr2=0x7ff1d40c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 --2- 172.21.15.8:0/2128949680 >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 -- 172.21.15.8:0/2128949680 >> v1:172.21.15.46:6789/0 conn(0x7ff1d415a3b0 legacy=0x7ff1d40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 -- 172.21.15.8:0/2128949680 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40c47e0 con 0x7ff1d4153c80 2023-10-30T12:45:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1d9b00640 1 --2- 172.21.15.8:0/2128949680 >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.769+0000 7ff1da301640 1 --2- 172.21.15.8:0/2128949680 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 0x7ff1d415cf90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097c0 tx=0x7ff1d001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b65bb0e9a223419 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1d92ff640 1 -- 172.21.15.8:0/2128949680 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00a8970 con 0x7ff1d4153c80 2023-10-30T12:45:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1d92ff640 1 -- 172.21.15.8:0/2128949680 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00a8ad0 con 0x7ff1d4153c80 2023-10-30T12:45:36.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1d92ff640 1 -- 172.21.15.8:0/2128949680 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00a8e60 con 0x7ff1d4153c80 2023-10-30T12:45:36.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- 172.21.15.8:0/2128949680 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 msgr2=0x7ff1d415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- 172.21.15.8:0/2128949680 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 0x7ff1d415cf90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097c0 tx=0x7ff1d001ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:36.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- 172.21.15.8:0/2128949680 shutdown_connections 2023-10-30T12:45:36.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- 172.21.15.8:0/2128949680 >> v2:172.21.15.46:3300/0 conn(0x7ff1d40c4a10 0x7ff1d40c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- 172.21.15.8:0/2128949680 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d4153c80 0x7ff1d415cf90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- 172.21.15.8:0/2128949680 >> 172.21.15.8:0/2128949680 conn(0x7ff1d40715c0 msgr2=0x7ff1d40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:36.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- 172.21.15.8:0/2128949680 shutdown_connections 2023-10-30T12:45:36.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- 172.21.15.8:0/2128949680 wait complete. 2023-10-30T12:45:36.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 Processor -- start 2023-10-30T12:45:36.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- start start 2023-10-30T12:45:36.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d40c4a10 0x7ff1d41f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d4153c80 0x7ff1d40c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d41511a0 con 0x7ff1d415a3b0 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4151480 con 0x7ff1d4153c80 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dc565640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4151760 con 0x7ff1d40c4a10 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.773+0000 7ff1dab02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56176/0 (socket says 172.21.15.8:56176) 2023-10-30T12:45:36.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 -- 172.21.15.8:0/3396791590 learned_addr learned my addr 172.21.15.8:0/3396791590 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:36.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1d9b00640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d4153c80 0x7ff1d40c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1da301640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d40c4a10 0x7ff1d41f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 -- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d40c4a10 msgr2=0x7ff1d41f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d40c4a10 0x7ff1d41f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 -- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d4153c80 msgr2=0x7ff1d40c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d4153c80 0x7ff1d40c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 -- 172.21.15.8:0/3396791590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40094d0 con 0x7ff1d415a3b0 2023-10-30T12:45:36.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dab02640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c730 tx=0x7ff1cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:36.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003b80 con 0x7ff1d415a3b0 2023-10-30T12:45:36.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc003ce0 con 0x7ff1d415a3b0 2023-10-30T12:45:36.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0009420 con 0x7ff1d415a3b0 2023-10-30T12:45:36.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00aa30 con 0x7ff1d415a3b0 2023-10-30T12:45:36.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d4151d20 con 0x7ff1d415a3b0 2023-10-30T12:45:36.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d404e170 con 0x7ff1d415a3b0 2023-10-30T12:45:36.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1cc00ab90 con 0x7ff1d415a3b0 2023-10-30T12:45:36.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.777+0000 7ff1caffd640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 0x7ff1b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:36.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.781+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1cc083b90 con 0x7ff1d415a3b0 2023-10-30T12:45:36.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.781+0000 7ff1da301640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 0x7ff1b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:36.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.781+0000 7ff1da301640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 0x7ff1b8068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00b2000 tx=0x7ff1d00aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:36.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.781+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1cc051020 con 0x7ff1d415a3b0 2023-10-30T12:45:36.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.973+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1d4157650 con 0x7ff1d415a3b0 2023-10-30T12:45:36.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.973+0000 7ff1caffd640 1 -- 172.21.15.8:0/3396791590 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7ff1cc054640 con 0x7ff1d415a3b0 2023-10-30T12:45:36.977 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:36.978 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:36.978 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:45:36.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 msgr2=0x7ff1b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 0x7ff1b8068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00b2000 tx=0x7ff1d00aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 msgr2=0x7ff1d414e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c730 tx=0x7ff1cc00cbf0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 shutdown_connections 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff1b80664a0 0x7ff1b8068960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d4153c80 0x7ff1d40c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff1d415a3b0 0x7ff1d414e030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 --2- 172.21.15.8:0/3396791590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff1d40c4a10 0x7ff1d41f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:36.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 >> 172.21.15.8:0/3396791590 conn(0x7ff1d40715c0 msgr2=0x7ff1d4158640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:36.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 shutdown_connections 2023-10-30T12:45:36.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:36.977+0000 7ff1dc565640 1 -- 172.21.15.8:0/3396791590 wait complete. 2023-10-30T12:45:37.399 INFO:tasks.ceph.mds.a.smithi008.stderr:2023-10-30T12:45:37.397+0000 7fe0693b9640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:45:37.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-30T12:45:18.463017+0000', 'modified': '2023-10-30T12:45:35.211861+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4303, 'mds_1': 4276, 'mds_2': 4247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4247': {'gid': 4247, 'name': 'c', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.8:6839/710307090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6836', 'nonce': 710307090}, {'type': 'v1', 'addr': '172.21.15.8:6839', 'nonce': 710307090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4276': {'gid': 4276, 'name': 'f', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.46:6833/3888673612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 3888673612}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 3888673612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4303': {'gid': 4303, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.8:6837/2926864154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6834', 'nonce': 2926864154}, {'type': 'v1', 'addr': '172.21.15.8:6837', 'nonce': 2926864154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-30T12:45:37.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-30T12:45:37.602 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-30T12:45:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 Processor -- start 2023-10-30T12:45:37.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 -- start start 2023-10-30T12:45:37.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f35ac06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac0c55f0 con 0x7f35ac06be90 2023-10-30T12:45:37.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac1567b0 con 0x7f35ac0c6320 2023-10-30T12:45:37.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b2f01640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c5f00 con 0x7f35ac14fc50 2023-10-30T12:45:37.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b149e640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f35ac06be90 0x7f35ac155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53964/0 (socket says 172.21.15.8:53964) 2023-10-30T12:45:37.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b149e640 1 -- 172.21.15.8:0/644537554 learned_addr learned my addr 172.21.15.8:0/644537554 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:37.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.757+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f35ac06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558335858 0 0) 0x7f35ac0c55f0 con 0x7f35ac06be90 2023-10-30T12:45:37.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f359c003450 con 0x7f35ac06be90 2023-10-30T12:45:37.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437097162 0 0) 0x7f35ac1567b0 con 0x7f35ac0c6320 2023-10-30T12:45:37.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac0c55f0 con 0x7f35ac0c6320 2023-10-30T12:45:37.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4058580316 0 0) 0x7f359c003450 con 0x7f35ac06be90 2023-10-30T12:45:37.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 msgr2=0x7f35ac06fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f35ac06fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 >> v1:172.21.15.46:6789/0 conn(0x7f35ac0c6320 legacy=0x7f35ac070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac156a70 con 0x7f35ac06be90 2023-10-30T12:45:37.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f35a4001a00 con 0x7f35ac06be90 2023-10-30T12:45:37.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 >> v1:172.21.15.8:6789/0 conn(0x7f35ac06be90 legacy=0x7f35ac155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f359c003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c008860 0x7f359c00ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f359c00d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c5f00 con 0x7f359c00b190 2023-10-30T12:45:37.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c00dd00 con 0x7f359c008860 2023-10-30T12:45:37.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c00dec0 con 0x7f35ac14fc50 2023-10-30T12:45:37.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f359c00d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f359c003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35abfff640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c008860 0x7f359c00ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 -- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 msgr2=0x7f359c003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac14fc50 0x7f359c003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 -- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c008860 msgr2=0x7f359c00ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c008860 0x7f359c00ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 -- 172.21.15.8:0/644537554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c003450 con 0x7f359c00b190 2023-10-30T12:45:37.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.761+0000 7f35b149e640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f359c00d580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35a400a7c0 tx=0x7f35a401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6c5ab6b3c1b3fc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:37.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40ad630 con 0x7f359c00b190 2023-10-30T12:45:37.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a40a8070 con 0x7f359c00b190 2023-10-30T12:45:37.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35ab7fe640 1 -- 172.21.15.8:0/644537554 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40ada30 con 0x7f359c00b190 2023-10-30T12:45:37.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- 172.21.15.8:0/644537554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 msgr2=0x7f359c00d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 --2- 172.21.15.8:0/644537554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f359c00d580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35a400a7c0 tx=0x7f35a401d2a0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b149e640 1 -- 172.21.15.8:0/644537554 reap_dead start 2023-10-30T12:45:37.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- 172.21.15.8:0/644537554 shutdown_connections 2023-10-30T12:45:37.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- 172.21.15.8:0/644537554 >> 172.21.15.8:0/644537554 conn(0x7f35ac0cac70 msgr2=0x7f35ac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:37.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- 172.21.15.8:0/644537554 shutdown_connections 2023-10-30T12:45:37.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- 172.21.15.8:0/644537554 wait complete. 2023-10-30T12:45:37.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 Processor -- start 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- start start 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac1e1a30 0x7f35ac1df000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac1df540 0x7f35ac1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac072030 con 0x7f359c00b190 2023-10-30T12:45:37.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1f2d60 con 0x7f35ac1e1a30 2023-10-30T12:45:37.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b2f01640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0728f0 con 0x7f35ac1df540 2023-10-30T12:45:37.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b0c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b0c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56182/0 (socket says 172.21.15.8:56182) 2023-10-30T12:45:37.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.765+0000 7f35b0c9d640 1 -- 172.21.15.8:0/3123891054 learned_addr learned my addr 172.21.15.8:0/3123891054 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:37.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35abfff640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac1e1a30 0x7f35ac1df000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b149e640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac1df540 0x7f35ac1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 -- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac1df540 msgr2=0x7f35ac1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac1df540 0x7f35ac1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 -- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac1e1a30 msgr2=0x7f35ac1df000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac1e1a30 0x7f35ac1df000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 -- 172.21.15.8:0/3123891054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac072bd0 con 0x7f359c00b190 2023-10-30T12:45:37.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35a00119d0 tx=0x7f35a0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:37.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35a97fa640 1 -- 172.21.15.8:0/3123891054 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000fb50 con 0x7f359c00b190 2023-10-30T12:45:37.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35a97fa640 1 -- 172.21.15.8:0/3123891054 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a000fcb0 con 0x7f359c00b190 2023-10-30T12:45:37.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac1f3040 con 0x7f359c00b190 2023-10-30T12:45:37.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35a97fa640 1 -- 172.21.15.8:0/3123891054 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0014440 con 0x7f359c00b190 2023-10-30T12:45:37.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac0724c0 con 0x7f359c00b190 2023-10-30T12:45:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35a97fa640 1 -- 172.21.15.8:0/3123891054 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f35a00145a0 con 0x7f359c00b190 2023-10-30T12:45:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.769+0000 7f35a97fa640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 0x7f3590068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35a97fa640 1 -- 172.21.15.8:0/3123891054 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a00848d0 con 0x7f359c00b190 2023-10-30T12:45:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 0x7f3590068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 Processor -- start 2023-10-30T12:45:37.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- start start 2023-10-30T12:45:37.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 0x7f3590068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3598005630 tx=0x7f3598009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:37.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d69bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b1dcc0 con 0x5629e7d6a130 2023-10-30T12:45:37.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b80920 con 0x5629e7d6aa50 2023-10-30T12:45:37.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b7de50 con 0x5629e7d69810 2023-10-30T12:45:37.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56184/0 (socket says 172.21.15.8:56184) 2023-10-30T12:45:37.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 -- 172.21.15.8:0/161515603 learned_addr learned my addr 172.21.15.8:0/161515603 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:37.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b149e640 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d69bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 -- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 msgr2=0x5629e7d69bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d69bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 -- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 msgr2=0x5629e7d74aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d74aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 -- 172.21.15.8:0/161515603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e7872890 con 0x5629e7d6a130 2023-10-30T12:45:37.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35abfff640 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35980164f0 tx=0x7f3598016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d903d2dfb2241b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35967fc640 1 -- 172.21.15.8:0/161515603 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359801b040 con 0x5629e7d6a130 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35967fc640 1 -- 172.21.15.8:0/161515603 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598015610 con 0x5629e7d6a130 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35967fc640 1 -- 172.21.15.8:0/161515603 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598015980 con 0x5629e7d6a130 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- 172.21.15.8:0/161515603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 msgr2=0x5629e7d6a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35980164f0 tx=0x7f3598016000 comp rx=0 tx=0).stop 2023-10-30T12:45:37.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- 172.21.15.8:0/161515603 shutdown_connections 2023-10-30T12:45:37.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d74aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d6a510 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 --2- 172.21.15.8:0/161515603 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d69bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- 172.21.15.8:0/161515603 >> 172.21.15.8:0/161515603 conn(0x5629e7d1ffc0 msgr2=0x5629e7d675d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:37.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- 172.21.15.8:0/161515603 shutdown_connections 2023-10-30T12:45:37.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.773+0000 7f35b628a000 1 -- 172.21.15.8:0/161515603 wait complete. 2023-10-30T12:45:37.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- start start 2023-10-30T12:45:37.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 Processor -- start 2023-10-30T12:45:37.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 0x5629e7d64490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b79740 con 0x5629e7d6aa50 2023-10-30T12:45:37.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b38d10 con 0x5629e7d6a130 2023-10-30T12:45:37.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b389b0 con 0x5629e7d69810 2023-10-30T12:45:37.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 10 client.0 _ll_register_callbacks cb 0x5629e7d14390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:37.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56200/0 (socket says 172.21.15.8:56200) 2023-10-30T12:45:37.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 -- 172.21.15.8:0/2752211387 learned_addr learned my addr 172.21.15.8:0/2752211387 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:37.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35abfff640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 0x5629e7d64490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 -- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 msgr2=0x5629e7d63f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d63f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 -- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 msgr2=0x5629e7d64490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 0x5629e7d64490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 -- 172.21.15.8:0/2752211387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359800f030 con 0x5629e7d6aa50 2023-10-30T12:45:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35abfff640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 0x5629e7d64490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b149e640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35a400a660 tx=0x7f35a401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d818210d4fe6e48d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40011d0 con 0x5629e7d6aa50 2023-10-30T12:45:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a4001800 con 0x5629e7d6aa50 2023-10-30T12:45:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629e7b2d370 con 0x5629e7d6aa50 2023-10-30T12:45:37.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a40019c0 con 0x5629e7d6aa50 2023-10-30T12:45:37.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 10 client.4644 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:37.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x5629e7b3b460 con 0x5629e7d6aa50 2023-10-30T12:45:37.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5629e7b2d370 con 0x5629e7d6aa50 2023-10-30T12:45:37.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f35a40b9460 con 0x5629e7d6aa50 2023-10-30T12:45:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 10 client.4644 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 10 client.4644 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-30T12:45:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 20 client.4644 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 4 client.4644 mds_command: new command op to 4247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.777+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 --> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629e7b132d0 con 0x5629e7d70790 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.781+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.781+0000 7f35b0c9d640 1 -- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 msgr2=0x5629e7d70b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.781+0000 7f35b0c9d640 1 -- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 msgr2=0x5629e7d70b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.781+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 crc :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:45:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.781+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:45:37.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.985+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:37.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.985+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0a538303b8623f8 server_cookie=337e4aa682b57fbf in_seq=0 out_seq=0 2023-10-30T12:45:37.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:37.985+0000 7f358b7fe640 10 client.4644 ms_handle_connect on v2:172.21.15.8:6836/710307090 2023-10-30T12:45:41.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f358b7fe640 1 -- 172.21.15.8:0/2752211387 <== mds.2 v2:172.21.15.8:6836/710307090 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629e7b132d0 con 0x5629e7d70790 2023-10-30T12:45:41.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f358b7fe640 10 client.4644 handle_command_reply: tid=0 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 client.4644 shutdown 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 msgr2=0x5629e7d6fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35a400a660 tx=0x7f35a401a040 comp rx=0 tx=0).stop 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 shutdown_connections 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7d70790 0x5629e7d70b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6a130 0x5629e7d64490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6aa50 0x5629e7d6fa20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 --2- 172.21.15.8:0/2752211387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d69810 0x5629e7d63f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f358b7fe640 0 client.4644 ms_handle_reset on v2:172.21.15.8:6836/710307090 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 >> 172.21.15.8:0/2752211387 conn(0x5629e7d1ffc0 msgr2=0x5629e7d69090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 shutdown_connections 2023-10-30T12:45:41.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.213+0000 7f35b628a000 1 -- 172.21.15.8:0/2752211387 wait complete. 2023-10-30T12:45:41.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.217+0000 7f35b628a000 20 client.4644 trim_cache size 0 max 16384 2023-10-30T12:45:41.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 Processor -- start 2023-10-30T12:45:41.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 -- start start 2023-10-30T12:45:41.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6a130 0x5629e7d801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d80720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7d7bd90 con 0x5629e7d69810 2023-10-30T12:45:41.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7833940 con 0x5629e7d6aa50 2023-10-30T12:45:41.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b750b0 con 0x5629e7d6a130 2023-10-30T12:45:41.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b0c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b0c9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53230/0 (socket says 172.21.15.8:53230) 2023-10-30T12:45:41.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b0c9d640 1 -- 172.21.15.8:0/669507329 learned_addr learned my addr 172.21.15.8:0/669507329 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35b149e640 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d80720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.233+0000 7f35abfff640 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6a130 0x5629e7d801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 -- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6a130 msgr2=0x5629e7d801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6a130 0x5629e7d801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 -- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 msgr2=0x5629e7d80720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d80720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 -- 172.21.15.8:0/669507329 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e7b21760 con 0x5629e7d69810 2023-10-30T12:45:41.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35a00556a0 tx=0x7f35a0082c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5d80b137e5dac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f3596ffd640 1 -- 172.21.15.8:0/669507329 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0053ae0 con 0x5629e7d69810 2023-10-30T12:45:41.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f3596ffd640 1 -- 172.21.15.8:0/669507329 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0082e70 con 0x5629e7d69810 2023-10-30T12:45:41.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f3596ffd640 1 -- 172.21.15.8:0/669507329 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00520b0 con 0x5629e7d69810 2023-10-30T12:45:41.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- 172.21.15.8:0/669507329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 msgr2=0x5629e7d840c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35a00556a0 tx=0x7f35a0082c30 comp rx=0 tx=0).stop 2023-10-30T12:45:41.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- 172.21.15.8:0/669507329 shutdown_connections 2023-10-30T12:45:41.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d6aa50 0x5629e7d80720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6a130 0x5629e7d801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- 172.21.15.8:0/669507329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d69810 0x5629e7d840c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- 172.21.15.8:0/669507329 >> 172.21.15.8:0/669507329 conn(0x5629e7d1ffc0 msgr2=0x5629e7d24750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- 172.21.15.8:0/669507329 shutdown_connections 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- 172.21.15.8:0/669507329 wait complete. 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- start start 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 Processor -- start 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d69810 0x5629e7d8d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 0x5629e7d8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b76b40 con 0x5629e7d6a130 2023-10-30T12:45:41.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e7d32020 con 0x5629e7d69810 2023-10-30T12:45:41.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.237+0000 7f35b628a000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e7b7d740 con 0x5629e7d6aa50 2023-10-30T12:45:41.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 10 client.0 _ll_register_callbacks cb 0x7f3598013f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:41.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53238/0 (socket says 172.21.15.8:53238) 2023-10-30T12:45:41.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 -- 172.21.15.8:0/993225367 learned_addr learned my addr 172.21.15.8:0/993225367 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d69810 0x5629e7d8d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b149e640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 0x5629e7d8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 -- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 msgr2=0x5629e7d8dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 0x5629e7d8dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 -- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d69810 msgr2=0x5629e7d8d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d69810 0x5629e7d8d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 -- 172.21.15.8:0/993225367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a0071040 con 0x5629e7d6a130 2023-10-30T12:45:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b149e640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 0x5629e7d8dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35abfff640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3598009010 tx=0x7f35980012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb6455b70304e51 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598023d80 con 0x5629e7d6a130 2023-10-30T12:45:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3598030400 con 0x5629e7d6a130 2023-10-30T12:45:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629e7b116f0 con 0x5629e7d6a130 2023-10-30T12:45:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35980305c0 con 0x5629e7d6a130 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 10 client.4650 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x5629e7872cc0 con 0x5629e7d6a130 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5629e7b116f0 con 0x5629e7d6a130 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f3598021040 con 0x5629e7d6a130 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 10 client.4650 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:41.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 10 client.4650 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-30T12:45:41.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 20 client.4650 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:41.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7dd2f30 0x5629e7dd5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 4 client.4650 mds_command: new command op to 4247 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:45:41.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 --> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5629e7b72720 con 0x5629e7dd2f30 2023-10-30T12:45:41.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7dd2f30 0x5629e7dd5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b0c9d640 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7dd2f30 0x5629e7dd5320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=346250ebf3d4e140 server_cookie=d0fe3d5d02e7babf in_seq=0 out_seq=0 2023-10-30T12:45:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 10 client.4650 ms_handle_connect on v2:172.21.15.8:6836/710307090 2023-10-30T12:45:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 1 -- 172.21.15.8:0/993225367 <== mds.2 v2:172.21.15.8:6836/710307090 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5629e7b72720 con 0x5629e7dd2f30 2023-10-30T12:45:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f358b7fe640 10 client.4650 handle_command_reply: tid=0 2023-10-30T12:45:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 client.4650 shutdown 2023-10-30T12:45:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 msgr2=0x5629e7d8d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.241+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3598009010 tx=0x7f35980012e0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 shutdown_connections 2023-10-30T12:45:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:6836/710307090,v1:172.21.15.8:6839/710307090] conn(0x5629e7dd2f30 0x5629e7dd5320 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5629e7d6aa50 0x5629e7d8dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629e7d6a130 0x5629e7d8d7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 --2- 172.21.15.8:0/993225367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629e7d69810 0x5629e7d8d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 >> 172.21.15.8:0/993225367 conn(0x5629e7d1ffc0 msgr2=0x5629e7d83850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f358b7fe640 0 client.4650 ms_handle_reset on v2:172.21.15.8:6836/710307090 2023-10-30T12:45:41.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 shutdown_connections 2023-10-30T12:45:41.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 1 -- 172.21.15.8:0/993225367 wait complete. 2023-10-30T12:45:41.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b628a000 20 client.4650 trim_cache size 0 max 16384 2023-10-30T12:45:41.260 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:45:41.260 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "d8d9e729-1294-467e-ac5d-e55aafd29b99", 2023-10-30T12:45:41.260 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 2, 2023-10-30T12:45:41.261 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 4247, 2023-10-30T12:45:41.261 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-10-30T12:45:41.261 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-10-30T12:45:41.261 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-10-30T12:45:41.262 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 11.056099264, 2023-10-30T12:45:41.262 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 12, 2023-10-30T12:45:41.262 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 20, 2023-10-30T12:45:41.262 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 20, 2023-10-30T12:45:41.262 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 29.874346284000001 2023-10-30T12:45:41.263 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:45:41.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 msgr2=0x7f3590068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 0x7f3590068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3598005630 tx=0x7f3598009040 comp rx=0 tx=0).stop 2023-10-30T12:45:41.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 msgr2=0x7f35ac1f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35a00119d0 tx=0x7f35a0011e90 comp rx=0 tx=0).stop 2023-10-30T12:45:41.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 shutdown_connections 2023-10-30T12:45:41.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f35900664a0 0x7f3590068960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac1e1a30 0x7f35ac1df000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f35ac1df540 0x7f35ac1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 --2- 172.21.15.8:0/3123891054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f359c00b190 0x7f35ac1f2400 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 >> 172.21.15.8:0/3123891054 conn(0x7f35ac0cac70 msgr2=0x7f35ac0c3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 shutdown_connections 2023-10-30T12:45:41.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.245+0000 7f35b2f01640 1 -- 172.21.15.8:0/3123891054 wait complete. 2023-10-30T12:45:41.271 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-30T12:45:41.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 Processor -- start 2023-10-30T12:45:41.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 -- start start 2023-10-30T12:45:41.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f08b015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b0159f20 con 0x7f08b0153c80 2023-10-30T12:45:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b015d620 con 0x7f08b00c42a0 2023-10-30T12:45:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b8301640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b015dec0 con 0x7f08b00c4a10 2023-10-30T12:45:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b609d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f08b0153c80 0x7f08b015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60418/0 (socket says 172.21.15.8:60418) 2023-10-30T12:45:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b609d640 1 -- 172.21.15.8:0/3938250259 learned_addr learned my addr 172.21.15.8:0/3938250259 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b589c640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f08b015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566155014 0 0) 0x7f08b0159f20 con 0x7f08b0153c80 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f089c003530 con 0x7f08b0153c80 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358543350 0 0) 0x7f089c003530 con 0x7f08b0153c80 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 msgr2=0x7f08b015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f08b015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 >> v1:172.21.15.46:6789/0 conn(0x7f08b00c42a0 legacy=0x7f08b00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b015d900 con 0x7f08b0153c80 2023-10-30T12:45:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f08ac001680 con 0x7f08b0153c80 2023-10-30T12:45:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 >> v1:172.21.15.8:6789/0 conn(0x7f08b0153c80 legacy=0x7f08b015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f089c0039b0 0x7f089c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0047d0 0x7f089c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f089c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b015dec0 con 0x7f089c0039b0 2023-10-30T12:45:41.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c00bad0 con 0x7f089c0047d0 2023-10-30T12:45:41.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.417+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c00bc90 con 0x7f08b00c4a10 2023-10-30T12:45:41.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f089c0039b0 0x7f089c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b589c640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0047d0 0x7f089c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b689e640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f089c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 -- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 msgr2=0x7f089c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f089c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 -- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0047d0 msgr2=0x7f089c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0047d0 0x7f089c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 -- 172.21.15.8:0/3938250259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c003530 con 0x7f089c0039b0 2023-10-30T12:45:41.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b689e640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b00c4a10 0x7f089c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:41.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b609d640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f089c0039b0 0x7f089c003dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c210 tx=0x7f08ac01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f974b40ad59a50 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac005ae0 con 0x7f089c0039b0 2023-10-30T12:45:41.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac0ab070 con 0x7f089c0039b0 2023-10-30T12:45:41.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.421+0000 7f08b509b640 1 -- 172.21.15.8:0/3938250259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac007070 con 0x7f089c0039b0 2023-10-30T12:45:41.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- 172.21.15.8:0/3938250259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f089c0039b0 msgr2=0x7f089c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 --2- 172.21.15.8:0/3938250259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f089c0039b0 0x7f089c003dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c210 tx=0x7f08ac01f350 comp rx=0 tx=0).stop 2023-10-30T12:45:41.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 -- 172.21.15.8:0/3938250259 reap_dead start 2023-10-30T12:45:41.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- 172.21.15.8:0/3938250259 shutdown_connections 2023-10-30T12:45:41.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- 172.21.15.8:0/3938250259 >> 172.21.15.8:0/3938250259 conn(0x7f08b00715c0 msgr2=0x7f08b00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- 172.21.15.8:0/3938250259 shutdown_connections 2023-10-30T12:45:41.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- 172.21.15.8:0/3938250259 wait complete. 2023-10-30T12:45:41.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 Processor -- start 2023-10-30T12:45:41.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- start start 2023-10-30T12:45:41.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0039b0 0x7f08b015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b0156e90 0x7f08b0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b015c1f0 con 0x7f08b01550b0 2023-10-30T12:45:41.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0154df0 con 0x7f089c0039b0 2023-10-30T12:45:41.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b8301640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00ce7f0 con 0x7f08b0156e90 2023-10-30T12:45:41.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53250/0 (socket says 172.21.15.8:53250) 2023-10-30T12:45:41.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 -- 172.21.15.8:0/1126805106 learned_addr learned my addr 172.21.15.8:0/1126805106 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b589c640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b0156e90 0x7f08b0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b609d640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0039b0 0x7f08b015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 -- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b0156e90 msgr2=0x7f08b0158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b0156e90 0x7f08b0158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 -- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0039b0 msgr2=0x7f08b015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0039b0 0x7f08b015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 -- 172.21.15.8:0/1126805106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00cea90 con 0x7f08b01550b0 2023-10-30T12:45:41.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.425+0000 7f08b689e640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08b008cc20 tx=0x7f08a800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 -- 172.21.15.8:0/1126805106 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8003b90 con 0x7f08b01550b0 2023-10-30T12:45:41.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 -- 172.21.15.8:0/1126805106 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8003cf0 con 0x7f08b01550b0 2023-10-30T12:45:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b00ced70 con 0x7f08b01550b0 2023-10-30T12:45:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 -- 172.21.15.8:0/1126805106 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8005310 con 0x7f08b01550b0 2023-10-30T12:45:41.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b015b9f0 con 0x7f08b01550b0 2023-10-30T12:45:41.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 -- 172.21.15.8:0/1126805106 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08a8005470 con 0x7f08b01550b0 2023-10-30T12:45:41.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 0x7f0894068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08a6ffd640 1 -- 172.21.15.8:0/1126805106 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a80837c0 con 0x7f08b01550b0 2023-10-30T12:45:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08b609d640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 0x7f0894068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.429+0000 7f08bb688000 1 Processor -- start 2023-10-30T12:45:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- start start 2023-10-30T12:45:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b609d640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 0x7f0894068990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0af000 tx=0x7f08ac0aa000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x5581662debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x5581662e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558166092cc0 con 0x5581662df130 2023-10-30T12:45:41.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660f5920 con 0x5581662dfa50 2023-10-30T12:45:41.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581660f2e50 con 0x5581662de810 2023-10-30T12:45:41.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53262/0 (socket says 172.21.15.8:53262) 2023-10-30T12:45:41.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 -- 172.21.15.8:0/3933847716 learned_addr learned my addr 172.21.15.8:0/3933847716 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b609d640 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x5581662debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b689e640 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x5581662e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 -- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 msgr2=0x5581662debf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x5581662debf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 -- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 msgr2=0x5581662e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x5581662e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 -- 172.21.15.8:0/3933847716 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558165de7890 con 0x5581662df130 2023-10-30T12:45:41.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08b589c640 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08a00099d0 tx=0x7f08a0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499cf6b799a089a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f0893fff640 1 -- 172.21.15.8:0/3933847716 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0013980 con 0x5581662df130 2023-10-30T12:45:41.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f0893fff640 1 -- 172.21.15.8:0/3933847716 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0013ae0 con 0x5581662df130 2023-10-30T12:45:41.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f0893fff640 1 -- 172.21.15.8:0/3933847716 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0013e50 con 0x5581662df130 2023-10-30T12:45:41.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- 172.21.15.8:0/3933847716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 msgr2=0x5581662df510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08a00099d0 tx=0x7f08a0009190 comp rx=0 tx=0).stop 2023-10-30T12:45:41.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- 172.21.15.8:0/3933847716 shutdown_connections 2023-10-30T12:45:41.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x5581662e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662df510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 --2- 172.21.15.8:0/3933847716 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x5581662debf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- 172.21.15.8:0/3933847716 >> 172.21.15.8:0/3933847716 conn(0x558166294fc0 msgr2=0x5581662dc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- 172.21.15.8:0/3933847716 shutdown_connections 2023-10-30T12:45:41.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- 172.21.15.8:0/3933847716 wait complete. 2023-10-30T12:45:41.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 -- start start 2023-10-30T12:45:41.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.433+0000 7f08bb688000 1 Processor -- start 2023-10-30T12:45:41.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662de810 0x5581662d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662dfa50 0x5581662d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660ee740 con 0x5581662df130 2023-10-30T12:45:41.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660add10 con 0x5581662de810 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581660ad9b0 con 0x5581662dfa50 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 10 client.0 _ll_register_callbacks cb 0x558166289390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53264/0 (socket says 172.21.15.8:53264) 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 -- 172.21.15.8:0/3489957965 learned_addr learned my addr 172.21.15.8:0/3489957965 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b609d640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662de810 0x5581662d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b689e640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662dfa50 0x5581662d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 -- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662dfa50 msgr2=0x5581662d95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662dfa50 0x5581662d95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 -- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662de810 msgr2=0x5581662d8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662de810 0x5581662d8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 -- 172.21.15.8:0/3489957965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a0009630 con 0x5581662df130 2023-10-30T12:45:41.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b589c640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08a001a000 tx=0x7f08a00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8551ffddffa4e478 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:41.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0004540 con 0x5581662df130 2023-10-30T12:45:41.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581660a2370 con 0x5581662df130 2023-10-30T12:45:41.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a0005bb0 con 0x5581662df130 2023-10-30T12:45:41.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a0005d70 con 0x5581662df130 2023-10-30T12:45:41.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 10 client.4662 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:41.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x5581660b0460 con 0x5581662df130 2023-10-30T12:45:41.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581660a2370 con 0x5581662df130 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f08a0013980 con 0x5581662df130 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 10 client.4662 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 10 client.4662 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 20 client.4662 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x5581662f2260 0x5581662f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 4 client.4662 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:45:41.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 --> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581660882d0 con 0x5581662f2260 2023-10-30T12:45:41.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.437+0000 7f08b609d640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x5581662f2260 0x5581662f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08b609d640 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x5581662f2260 0x5581662f4650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c39dd86651793a3 server_cookie=9ef6a24aa859b948 in_seq=0 out_seq=0 2023-10-30T12:45:41.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08917fa640 10 client.4662 ms_handle_connect on v2:172.21.15.46:6832/3888673612 2023-10-30T12:45:41.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08917fa640 1 -- 172.21.15.8:0/3489957965 <== mds.1 v2:172.21.15.46:6832/3888673612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5581660882d0 con 0x5581662f2260 2023-10-30T12:45:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08917fa640 10 client.4662 handle_command_reply: tid=0 2023-10-30T12:45:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 client.4662 shutdown 2023-10-30T12:45:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 msgr2=0x5581662d90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08a001a000 tx=0x7f08a00046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 shutdown_connections 2023-10-30T12:45:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x5581662f2260 0x5581662f4650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662dfa50 0x5581662d95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x5581662d90a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 --2- 172.21.15.8:0/3489957965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662de810 0x5581662d8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08917fa640 0 client.4662 ms_handle_reset on v2:172.21.15.46:6832/3888673612 2023-10-30T12:45:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 >> 172.21.15.8:0/3489957965 conn(0x558166294fc0 msgr2=0x5581662de090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 shutdown_connections 2023-10-30T12:45:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 1 -- 172.21.15.8:0/3489957965 wait complete. 2023-10-30T12:45:41.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.441+0000 7f08bb688000 20 client.4662 trim_cache size 0 max 16384 2023-10-30T12:45:41.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 Processor -- start 2023-10-30T12:45:41.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 -- start start 2023-10-30T12:45:41.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x55816633e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558165da8940 con 0x5581662df130 2023-10-30T12:45:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660ea0b0 con 0x5581662dfa50 2023-10-30T12:45:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581660ed5d0 con 0x5581662de810 2023-10-30T12:45:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53266/0 (socket says 172.21.15.8:53266) 2023-10-30T12:45:41.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 -- 172.21.15.8:0/2824800306 learned_addr learned my addr 172.21.15.8:0/2824800306 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b609d640 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b689e640 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x55816633e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 -- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 msgr2=0x55816633de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 -- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 msgr2=0x55816633e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x55816633e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 -- 172.21.15.8:0/2824800306 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581662f0f50 con 0x5581662df130 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08b589c640 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006f10 tx=0x7f08a0022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2993f6443bcaa99a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08a4ff9640 1 -- 172.21.15.8:0/2824800306 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0013ac0 con 0x5581662df130 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08a4ff9640 1 -- 172.21.15.8:0/2824800306 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0005bb0 con 0x5581662df130 2023-10-30T12:45:41.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.457+0000 7f08a4ff9640 1 -- 172.21.15.8:0/2824800306 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0033470 con 0x5581662df130 2023-10-30T12:45:41.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2824800306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 msgr2=0x55816633e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006f10 tx=0x7f08a0022040 comp rx=0 tx=0).stop 2023-10-30T12:45:41.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2824800306 shutdown_connections 2023-10-30T12:45:41.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662dfa50 0x55816633e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662df130 0x55816633e340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- 172.21.15.8:0/2824800306 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2824800306 >> 172.21.15.8:0/2824800306 conn(0x558166294fc0 msgr2=0x5581662e4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2824800306 shutdown_connections 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2824800306 wait complete. 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- start start 2023-10-30T12:45:41.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 Processor -- start 2023-10-30T12:45:41.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662df130 0x55816633bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660c4a10 con 0x5581662dfa50 2023-10-30T12:45:41.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581660ebb40 con 0x5581662df130 2023-10-30T12:45:41.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581660f12a0 con 0x5581662de810 2023-10-30T12:45:41.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 10 client.0 _ll_register_callbacks cb 0x7f08a00094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:41.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53268/0 (socket says 172.21.15.8:53268) 2023-10-30T12:45:41.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 -- 172.21.15.8:0/2827994201 learned_addr learned my addr 172.21.15.8:0/2827994201 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b589c640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662df130 0x55816633bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 -- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 msgr2=0x55816633ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 -- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662df130 msgr2=0x55816633bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662df130 0x55816633bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 -- 172.21.15.8:0/2827994201 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581660b0460 con 0x5581662dfa50 2023-10-30T12:45:41.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08b689e640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08a8054760 tx=0x7f08a8054ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3404b799738b8dc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:41.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8058920 con 0x5581662dfa50 2023-10-30T12:45:41.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a805cb60 con 0x5581662dfa50 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581662f0950 con 0x5581662dfa50 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a8005720 con 0x5581662dfa50 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 10 client.4668 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x558166096760 con 0x5581662dfa50 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581662f0950 con 0x5581662dfa50 2023-10-30T12:45:41.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f08a8050ee0 con 0x5581662dfa50 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 10 client.4668 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 10 client.4668 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 20 client.4668 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.461+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08bb688000 4 client.4668 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:45:41.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 --> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5581660866f0 con 0x558166349020 2023-10-30T12:45:41.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08b609d640 1 -- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 msgr2=0x55816633d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2023-10-30T12:45:41.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08b609d640 1 -- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 msgr2=0x55816633d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-30T12:45:41.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-30T12:45:41.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.465+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-30T12:45:41.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b609d640 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c9a8824f8580157 server_cookie=e256e784659b3275 in_seq=0 out_seq=0 2023-10-30T12:45:41.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08917fa640 10 client.4668 ms_handle_connect on v2:172.21.15.46:6832/3888673612 2023-10-30T12:45:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08917fa640 1 -- 172.21.15.8:0/2827994201 <== mds.1 v2:172.21.15.46:6832/3888673612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5581660866f0 con 0x558166349020 2023-10-30T12:45:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08917fa640 10 client.4668 handle_command_reply: tid=0 2023-10-30T12:45:41.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 client.4668 shutdown 2023-10-30T12:45:41.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 msgr2=0x55816633c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08a8054760 tx=0x7f08a8054ed0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 shutdown_connections 2023-10-30T12:45:41.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:6832/3888673612,v1:172.21.15.46:6833/3888673612] conn(0x558166349020 0x55816633d820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5581662df130 0x55816633bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5581662dfa50 0x55816633c960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08917fa640 0 client.4668 ms_handle_reset on v2:172.21.15.46:6832/3888673612 2023-10-30T12:45:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 --2- 172.21.15.8:0/2827994201 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5581662de810 0x55816633ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 >> 172.21.15.8:0/2827994201 conn(0x558166294fc0 msgr2=0x5581662e1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 shutdown_connections 2023-10-30T12:45:41.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 1 -- 172.21.15.8:0/2827994201 wait complete. 2023-10-30T12:45:41.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08bb688000 20 client.4668 trim_cache size 0 max 16384 2023-10-30T12:45:41.676 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:45:41.676 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "d8d9e729-1294-467e-ac5d-e55aafd29b99", 2023-10-30T12:45:41.676 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 1, 2023-10-30T12:45:41.677 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 4276, 2023-10-30T12:45:41.677 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-10-30T12:45:41.677 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-10-30T12:45:41.677 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-10-30T12:45:41.677 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 17.531922666, 2023-10-30T12:45:41.678 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 12, 2023-10-30T12:45:41.678 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 20, 2023-10-30T12:45:41.678 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 20, 2023-10-30T12:45:41.678 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 30.315065616999998 2023-10-30T12:45:41.678 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:45:41.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 msgr2=0x7f0894068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 0x7f0894068990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0af000 tx=0x7f08ac0aa000 comp rx=0 tx=0).stop 2023-10-30T12:45:41.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 msgr2=0x7f08b0155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08b008cc20 tx=0x7f08a800ead0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 shutdown_connections 2023-10-30T12:45:41.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f08940664d0 0x7f0894068990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f08b01550b0 0x7f08b0155560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f08b0156e90 0x7f08b0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 --2- 172.21.15.8:0/1126805106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f089c0039b0 0x7f08b015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 >> 172.21.15.8:0/1126805106 conn(0x7f08b00715c0 msgr2=0x7f08b0150bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 shutdown_connections 2023-10-30T12:45:41.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.669+0000 7f08b8301640 1 -- 172.21.15.8:0/1126805106 wait complete. 2023-10-30T12:45:41.693 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-30T12:45:41.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.833+0000 7f508f009640 1 Processor -- start 2023-10-30T12:45:41.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.833+0000 7f508f009640 1 -- start start 2023-10-30T12:45:41.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508f009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f5088065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508f009640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508806e390 con 0x7f5088070c30 2023-10-30T12:45:41.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508f009640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50880660e0 con 0x7f5088159970 2023-10-30T12:45:41.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508f009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5088072390 con 0x7f5088153c10 2023-10-30T12:45:41.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f5087fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f5088070c30 0x7f5088071070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60434/0 (socket says 172.21.15.8:60434) 2023-10-30T12:45:41.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f5087fff640 1 -- 172.21.15.8:0/2513078590 learned_addr learned my addr 172.21.15.8:0/2513078590 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:41.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f5088065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100868798 0 0) 0x7f508806e390 con 0x7f5088070c30 2023-10-30T12:45:41.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5078003530 con 0x7f5088070c30 2023-10-30T12:45:41.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2385548310 0 0) 0x7f5078003530 con 0x7f5088070c30 2023-10-30T12:45:41.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 msgr2=0x7f5088065ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f5088065ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 >> v1:172.21.15.46:6789/0 conn(0x7f5088159970 legacy=0x7f5088159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880bec10 con 0x7f5088070c30 2023-10-30T12:45:41.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f5074001680 con 0x7f5088070c30 2023-10-30T12:45:41.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 >> v1:172.21.15.8:6789/0 conn(0x7f5088070c30 legacy=0x7f5088071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50780039b0 0x7f5078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780047d0 0x7f5078004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f50780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088072390 con 0x7f50780039b0 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507800bac0 con 0x7f50780047d0 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507800bc80 con 0x7f5088153c10 2023-10-30T12:45:41.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50780039b0 0x7f5078003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f5087fff640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780047d0 0x7f5078004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508d5a6640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f50780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 -- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 msgr2=0x7f50780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5088153c10 0x7f50780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 -- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780047d0 msgr2=0x7f5078004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780047d0 0x7f5078004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.837+0000 7f508cda5640 1 -- 172.21.15.8:0/2513078590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078003530 con 0x7f50780039b0 2023-10-30T12:45:41.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508cda5640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50780039b0 0x7f5078003dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ae0 tx=0x7f507c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5046e639fc4f93b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0a9400 con 0x7f50780039b0 2023-10-30T12:45:41.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c0a9560 con 0x7f50780039b0 2023-10-30T12:45:41.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f50877fe640 1 -- 172.21.15.8:0/2513078590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0a9710 con 0x7f50780039b0 2023-10-30T12:45:41.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- 172.21.15.8:0/2513078590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50780039b0 msgr2=0x7f5078003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 --2- 172.21.15.8:0/2513078590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50780039b0 0x7f5078003dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ae0 tx=0x7f507c01ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:41.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508d5a6640 1 -- 172.21.15.8:0/2513078590 reap_dead start 2023-10-30T12:45:41.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- 172.21.15.8:0/2513078590 shutdown_connections 2023-10-30T12:45:41.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- 172.21.15.8:0/2513078590 >> 172.21.15.8:0/2513078590 conn(0x7f50880c7f60 msgr2=0x7f50880c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- 172.21.15.8:0/2513078590 shutdown_connections 2023-10-30T12:45:41.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- 172.21.15.8:0/2513078590 wait complete. 2023-10-30T12:45:41.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 Processor -- start 2023-10-30T12:45:41.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.841+0000 7f508f009640 1 -- start start 2023-10-30T12:45:41.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780039b0 0x7f5088150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f508814bd60 0x7f508815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088152c00 con 0x7f508815b5f0 2023-10-30T12:45:41.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508815c980 con 0x7f50780039b0 2023-10-30T12:45:41.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508814b920 con 0x7f508814bd60 2023-10-30T12:45:41.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53284/0 (socket says 172.21.15.8:53284) 2023-10-30T12:45:41.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 -- 172.21.15.8:0/3074281425 learned_addr learned my addr 172.21.15.8:0/3074281425 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508cda5640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780039b0 0x7f5088150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508d5a6640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f508814bd60 0x7f508815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 -- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f508814bd60 msgr2=0x7f508815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f508814bd60 0x7f508815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 -- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780039b0 msgr2=0x7f5088150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780039b0 0x7f5088150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 -- 172.21.15.8:0/3074281425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088152640 con 0x7f508815b5f0 2023-10-30T12:45:41.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f5087fff640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5074007290 tx=0x7f5074007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f50857fa640 1 -- 172.21.15.8:0/3074281425 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507400f8c0 con 0x7f508815b5f0 2023-10-30T12:45:41.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f50857fa640 1 -- 172.21.15.8:0/3074281425 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507400fa60 con 0x7f508815b5f0 2023-10-30T12:45:41.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f50857fa640 1 -- 172.21.15.8:0/3074281425 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507400fe10 con 0x7f508815b5f0 2023-10-30T12:45:41.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5088152920 con 0x7f508815b5f0 2023-10-30T12:45:41.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.845+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508814ca40 con 0x7f508815b5f0 2023-10-30T12:45:41.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f50857fa640 1 -- 172.21.15.8:0/3074281425 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f507401d460 con 0x7f508815b5f0 2023-10-30T12:45:41.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f50857fa640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 0x7f506c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f50857fa640 1 -- 172.21.15.8:0/3074281425 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5074084260 con 0x7f508815b5f0 2023-10-30T12:45:41.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508cda5640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 0x7f506c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 Processor -- start 2023-10-30T12:45:41.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 -- start start 2023-10-30T12:45:41.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508cda5640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 0x7f506c068990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f507c0ae000 tx=0x7f507c0b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af579cc0 con 0x5575af7c6a50 2023-10-30T12:45:41.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af5dc920 con 0x5575af7c5810 2023-10-30T12:45:41.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575af5d9e50 con 0x5575af7c6130 2023-10-30T12:45:41.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53296/0 (socket says 172.21.15.8:53296) 2023-10-30T12:45:41.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 -- 172.21.15.8:0/3915384449 learned_addr learned my addr 172.21.15.8:0/3915384449 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f5087fff640 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508cda5640 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 -- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 msgr2=0x5575af7c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 -- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 msgr2=0x5575af7c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.849+0000 7f508d5a6640 1 -- 172.21.15.8:0/3915384449 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575af2ce890 con 0x5575af7c6a50 2023-10-30T12:45:41.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f508806eeb0 tx=0x7f5080005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429b11f09b055e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f50727fc640 1 -- 172.21.15.8:0/3915384449 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080001820 con 0x5575af7c6a50 2023-10-30T12:45:41.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f50727fc640 1 -- 172.21.15.8:0/3915384449 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5080001980 con 0x5575af7c6a50 2023-10-30T12:45:41.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f50727fc640 1 -- 172.21.15.8:0/3915384449 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080001cf0 con 0x5575af7c6a50 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- 172.21.15.8:0/3915384449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 msgr2=0x5575af7d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f508806eeb0 tx=0x7f5080005420 comp rx=0 tx=0).stop 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- 172.21.15.8:0/3915384449 shutdown_connections 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7d0aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- 172.21.15.8:0/3915384449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- 172.21.15.8:0/3915384449 >> 172.21.15.8:0/3915384449 conn(0x5575af77bfc0 msgr2=0x5575af7c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- 172.21.15.8:0/3915384449 shutdown_connections 2023-10-30T12:45:41.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- 172.21.15.8:0/3915384449 wait complete. 2023-10-30T12:45:41.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- start start 2023-10-30T12:45:41.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 Processor -- start 2023-10-30T12:45:41.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 0x5575af7c0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af5d5740 con 0x5575af7c6a50 2023-10-30T12:45:41.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af594d10 con 0x5575af7c6130 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575af5949b0 con 0x5575af7c5810 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5092397000 10 client.0 _ll_register_callbacks cb 0x5575af770390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53306/0 (socket says 172.21.15.8:53306) 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 -- 172.21.15.8:0/2370166853 learned_addr learned my addr 172.21.15.8:0/2370166853 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508cda5640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5087fff640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 0x5575af7c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 -- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 msgr2=0x5575af7bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 -- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 msgr2=0x5575af7c0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 0x5575af7c0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 -- 172.21.15.8:0/2370166853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508000f040 con 0x5575af7c6a50 2023-10-30T12:45:41.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f5087fff640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 0x5575af7c0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:41.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f508d5a6640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5080001360 tx=0x7f5080001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d8b65c042eeab server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.853+0000 7f50677fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:41.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50800020d0 con 0x5575af7c6a50 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f508001f410 con 0x5575af7c6a50 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575af589370 con 0x5575af7c6a50 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f508001f5d0 con 0x5575af7c6a50 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 10 client.4680 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x5575af597460 con 0x5575af7c6a50 2023-10-30T12:45:41.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5575af589370 con 0x5575af7c6a50 2023-10-30T12:45:41.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f508001f730 con 0x5575af7c6a50 2023-10-30T12:45:41.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 10 client.4680 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:41.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 10 client.4680 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-30T12:45:41.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 20 client.4680 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:41.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af7cc9a0 0x5575af7ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 4 client.4680 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 --> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575af56f2d0 con 0x5575af7cc9a0 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f508cda5640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af7cc9a0 0x5575af7ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f508cda5640 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af7cc9a0 0x5575af7ccd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6ec5c17b399c7d server_cookie=5fe31976edaeaa5a in_seq=0 out_seq=0 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 10 client.4680 ms_handle_connect on v2:172.21.15.8:6834/2926864154 2023-10-30T12:45:41.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 1 -- 172.21.15.8:0/2370166853 <== mds.0 v2:172.21.15.8:6834/2926864154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575af56f2d0 con 0x5575af7cc9a0 2023-10-30T12:45:41.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 10 client.4680 handle_command_reply: tid=0 2023-10-30T12:45:41.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 client.4680 shutdown 2023-10-30T12:45:41.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 msgr2=0x5575af7cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5080001360 tx=0x7f5080001520 comp rx=0 tx=0).stop 2023-10-30T12:45:41.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 shutdown_connections 2023-10-30T12:45:41.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af7cc9a0 0x5575af7ccd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6130 0x5575af7c0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7cbc30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f50677fe640 0 client.4680 ms_handle_reset on v2:172.21.15.8:6834/2926864154 2023-10-30T12:45:41.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 --2- 172.21.15.8:0/2370166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 >> 172.21.15.8:0/2370166853 conn(0x5575af77bfc0 msgr2=0x5575af7c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 shutdown_connections 2023-10-30T12:45:41.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 1 -- 172.21.15.8:0/2370166853 wait complete. 2023-10-30T12:45:41.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.857+0000 7f5092397000 20 client.4680 trim_cache size 0 max 16384 2023-10-30T12:45:41.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 Processor -- start 2023-10-30T12:45:41.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 -- start start 2023-10-30T12:45:41.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 0x5575af7dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af28f940 con 0x5575af7c6130 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af5d43f0 con 0x5575af7c6a50 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575af5d10b0 con 0x5575af7c5810 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53314/0 (socket says 172.21.15.8:53314) 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 -- 172.21.15.8:0/703978859 learned_addr learned my addr 172.21.15.8:0/703978859 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f508cda5640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f508d5a6640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 0x5575af7dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 -- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 msgr2=0x5575af7e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 -- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 msgr2=0x5575af7dc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 0x5575af7dc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f5087fff640 1 -- 172.21.15.8:0/703978859 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575af7d7f50 con 0x5575af7c6130 2023-10-30T12:45:41.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f508d5a6640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 0x5575af7dc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:41.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.873+0000 7f508cda5640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:45:41.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5087fff640 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5074054f70 tx=0x7f5074015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c5d8e5092f3fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5072ffd640 1 -- 172.21.15.8:0/703978859 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074084070 con 0x5575af7c6130 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5072ffd640 1 -- 172.21.15.8:0/703978859 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074059180 con 0x5575af7c6130 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5072ffd640 1 -- 172.21.15.8:0/703978859 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074055270 con 0x5575af7c6130 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- 172.21.15.8:0/703978859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 msgr2=0x5575af7dc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5074054f70 tx=0x7f5074015040 comp rx=0 tx=0).stop 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- 172.21.15.8:0/703978859 shutdown_connections 2023-10-30T12:45:41.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c6a50 0x5575af7dc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6130 0x5575af7dc3f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- 172.21.15.8:0/703978859 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c5810 0x5575af7e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- 172.21.15.8:0/703978859 >> 172.21.15.8:0/703978859 conn(0x5575af77bfc0 msgr2=0x5575af78a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- 172.21.15.8:0/703978859 shutdown_connections 2023-10-30T12:45:41.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- 172.21.15.8:0/703978859 wait complete. 2023-10-30T12:45:41.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- start start 2023-10-30T12:45:41.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 Processor -- start 2023-10-30T12:45:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af5944d0 con 0x5575af7c6a50 2023-10-30T12:45:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575af616ba0 con 0x5575af7c5810 2023-10-30T12:45:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575af5cfc20 con 0x5575af7c6130 2023-10-30T12:45:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5092397000 10 client.0 _ll_register_callbacks cb 0x7f5080003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-30T12:45:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53328/0 (socket says 172.21.15.8:53328) 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 -- 172.21.15.8:0/133092926 learned_addr learned my addr 172.21.15.8:0/133092926 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508cda5640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f5087fff640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 -- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 msgr2=0x5575af7e9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7e9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 -- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 msgr2=0x5575af7e9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 -- 172.21.15.8:0/133092926 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508814ca40 con 0x5575af7c6a50 2023-10-30T12:45:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508cda5640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f508d5a6640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f508001fc00 tx=0x7f5080014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec32827a1f82fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.877+0000 7f50677fe640 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-10-30T12:45:41.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508001f820 con 0x5575af7c6a50 2023-10-30T12:45:41.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5080007e40 con 0x5575af7c6a50 2023-10-30T12:45:41.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575af56d6f0 con 0x5575af7c6a50 2023-10-30T12:45:41.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5080028440 con 0x5575af7c6a50 2023-10-30T12:45:41.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 10 client.4686 fetch_fsmap learned FSMap version 12 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x5575af57d760 con 0x5575af7c6a50 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5575af56d6f0 con 0x5575af7c6a50 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 12) v1 ==== 4468+0+0 (secure 0 0 0) 0x7f50800285a0 con 0x5575af7c6a50 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 10 client.4686 fetch_fsmap finished waiting for FSMap version 12 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 10 client.4686 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-30T12:45:41.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 20 client.4686 populate_metadata read hostname 'smithi008' 2023-10-30T12:45:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af82f140 0x5575af831530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 4 client.4686 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-30T12:45:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 --> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5575af5ce720 con 0x5575af82f140 2023-10-30T12:45:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508cda5640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af82f140 0x5575af831530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 10 client.4686 ms_handle_connect on v2:172.21.15.8:6834/2926864154 2023-10-30T12:45:41.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508cda5640 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af82f140 0x5575af831530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bda39f3d8f67c7 server_cookie=ba90ced107b387f9 in_seq=0 out_seq=0 2023-10-30T12:45:41.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 1 -- 172.21.15.8:0/133092926 <== mds.0 v2:172.21.15.8:6834/2926864154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5575af5ce720 con 0x5575af82f140 2023-10-30T12:45:41.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 10 client.4686 handle_command_reply: tid=0 2023-10-30T12:45:41.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 client.4686 shutdown 2023-10-30T12:45:41.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 msgr2=0x5575af7ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f508001fc00 tx=0x7f5080014000 comp rx=0 tx=0).stop 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 shutdown_connections 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:6834/2926864154,v1:172.21.15.8:6837/2926864154] conn(0x5575af82f140 0x5575af831530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5575af7c6a50 0x5575af7ea100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x5575af7c6130 0x5575af7e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 --2- 172.21.15.8:0/133092926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575af7c5810 0x5575af7e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 >> 172.21.15.8:0/133092926 conn(0x5575af77bfc0 msgr2=0x5575af789c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f50677fe640 0 client.4686 ms_handle_reset on v2:172.21.15.8:6834/2926864154 2023-10-30T12:45:41.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 shutdown_connections 2023-10-30T12:45:41.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 1 -- 172.21.15.8:0/133092926 wait complete. 2023-10-30T12:45:41.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f5092397000 20 client.4686 trim_cache size 0 max 16384 2023-10-30T12:45:41.900 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:45:41.900 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "d8d9e729-1294-467e-ac5d-e55aafd29b99", 2023-10-30T12:45:41.900 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2023-10-30T12:45:41.900 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 4303, 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 22.818581434999999, 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 12, 2023-10-30T12:45:41.901 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 20, 2023-10-30T12:45:41.902 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 20, 2023-10-30T12:45:41.902 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 30.512823739000002 2023-10-30T12:45:41.902 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:45:41.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 msgr2=0x7f506c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 0x7f506c068990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f507c0ae000 tx=0x7f507c0b3040 comp rx=0 tx=0).stop 2023-10-30T12:45:41.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 msgr2=0x7f508815bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:41.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5074007290 tx=0x7f5074007750 comp rx=0 tx=0).stop 2023-10-30T12:45:41.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 shutdown_connections 2023-10-30T12:45:41.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f506c0664d0 0x7f506c068990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f508814bd60 0x7f508815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f508815b5f0 0x7f508815bf00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 --2- 172.21.15.8:0/3074281425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50780039b0 0x7f5088150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:41.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 >> 172.21.15.8:0/3074281425 conn(0x7f50880c7f60 msgr2=0x7f5088158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:41.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 shutdown_connections 2023-10-30T12:45:41.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:41.881+0000 7f508f009640 1 -- 172.21.15.8:0/3074281425 wait complete. 2023-10-30T12:45:41.906 INFO:teuthology.run_tasks:Running task kclient... 2023-10-30T12:45:41.934 INFO:tasks.kclient:Mounting kernel clients... 2023-10-30T12:45:41.934 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-30T12:45:41.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:41.935 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-10-30T12:45:41.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:41.946 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2023-10-30T12:45:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:Cannot find device "ceph-brx" 2023-10-30T12:45:42.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:45:42.020 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-30T12:45:42.020 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-30T12:45:42.020 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:42.021 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-30T12:45:42.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.201+0000 7f0266303640 1 Processor -- start 2023-10-30T12:45:42.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.201+0000 7f0266303640 1 -- start start 2023-10-30T12:45:42.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0266303640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f026015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0266303640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f026015ac70 0x7f026015a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0266303640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026015dcc0 con 0x7f026006aeb0 2023-10-30T12:45:42.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0266303640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0260159800 con 0x7f026015ac70 2023-10-30T12:45:42.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0266303640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0260159d80 con 0x7f02600c8eb0 2023-10-30T12:45:42.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265b02640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f026006aeb0 0x7f0260069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60450/0 (socket says 172.21.15.8:60450) 2023-10-30T12:45:42.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265b02640 1 -- 172.21.15.8:0/2784997431 learned_addr learned my addr 172.21.15.8:0/2784997431 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:42.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0264b00640 1 --2- 172.21.15.8:0/2784997431 >> v2:172.21.15.46:3300/0 conn(0x7f026015ac70 0x7f026015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f026015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243139100 0 0) 0x7f02580014a0 con 0x7f026006aeb0 2023-10-30T12:45:42.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f024c003530 con 0x7f026006aeb0 2023-10-30T12:45:42.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961326893 0 0) 0x7f024c003530 con 0x7f026006aeb0 2023-10-30T12:45:42.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 msgr2=0x7f026015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:42.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f026015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 >> v2:172.21.15.46:3300/0 conn(0x7f026015ac70 msgr2=0x7f026015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:42.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 --2- 172.21.15.8:0/2784997431 >> v2:172.21.15.46:3300/0 conn(0x7f026015ac70 0x7f026015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026015a750 con 0x7f026006aeb0 2023-10-30T12:45:42.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f0258001b90 con 0x7f026006aeb0 2023-10-30T12:45:42.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 >> v1:172.21.15.8:6789/0 conn(0x7f026006aeb0 legacy=0x7f0260069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f024c003980 0x7f024c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c0047d0 0x7f024c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f024c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0260159800 con 0x7f024c003980 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0260159d80 con 0x7f024c0047d0 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c00b990 con 0x7f02600c8eb0 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f024c003980 0x7f024c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265b02640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f024c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0264b00640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c0047d0 0x7f024c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 -- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 msgr2=0x7f024c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:42.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600c8eb0 0x7f024c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 -- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c0047d0 msgr2=0x7f024c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:42.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c0047d0 0x7f024c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.205+0000 7f0265301640 1 -- 172.21.15.8:0/2784997431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c003530 con 0x7f024c003980 2023-10-30T12:45:42.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0265301640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f024c003980 0x7f024c003da0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f025c009ae0 tx=0x7f025c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c894d824263a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:42.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c0a5560 con 0x7f024c003980 2023-10-30T12:45:42.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c0a56c0 con 0x7f024c003980 2023-10-30T12:45:42.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0257fff640 1 -- 172.21.15.8:0/2784997431 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c0a5a70 con 0x7f024c003980 2023-10-30T12:45:42.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 -- 172.21.15.8:0/2784997431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f024c003980 msgr2=0x7f024c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 --2- 172.21.15.8:0/2784997431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f024c003980 0x7f024c003da0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f025c009ae0 tx=0x7f025c01ca50 comp rx=0 tx=0).stop 2023-10-30T12:45:42.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0265b02640 1 -- 172.21.15.8:0/2784997431 reap_dead start 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 -- 172.21.15.8:0/2784997431 shutdown_connections 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 -- 172.21.15.8:0/2784997431 >> 172.21.15.8:0/2784997431 conn(0x7f02600cef00 msgr2=0x7f02600cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 -- 172.21.15.8:0/2784997431 shutdown_connections 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.209+0000 7f0266303640 1 -- 172.21.15.8:0/2784997431 wait complete. 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 Processor -- start 2023-10-30T12:45:42.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- start start 2023-10-30T12:45:42.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c003980 0x7f02601f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600b3970 0x7f02600c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601545d0 con 0x7f02600c4990 2023-10-30T12:45:42.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02600c4500 con 0x7f024c003980 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600c5380 con 0x7f02600b3970 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53334/0 (socket says 172.21.15.8:53334) 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 -- 172.21.15.8:0/1876594837 learned_addr learned my addr 172.21.15.8:0/1876594837 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0264b00640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600b3970 0x7f02600c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265301640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c003980 0x7f02601f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 -- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600b3970 msgr2=0x7f02600c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600b3970 0x7f02600c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 -- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c003980 msgr2=0x7f02601f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c003980 0x7f02601f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 -- 172.21.15.8:0/1876594837 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600c71c0 con 0x7f02600c4990 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0265b02640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0258000f90 tx=0x7f0258011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800f920 con 0x7f02600c4990 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025800fac0 con 0x7f02600c4990 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026015c660 con 0x7f02600c4990 2023-10-30T12:45:42.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800fe70 con 0x7f02600c4990 2023-10-30T12:45:42.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026015cc20 con 0x7f02600c4990 2023-10-30T12:45:42.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.213+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026004e170 con 0x7f02600c4990 2023-10-30T12:45:42.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.217+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f025801f460 con 0x7f02600c4990 2023-10-30T12:45:42.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.217+0000 7f0255ffb640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 0x7f0244068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:42.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.217+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0258086d70 con 0x7f02600c4990 2023-10-30T12:45:42.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.217+0000 7f0265301640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 0x7f0244068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:42.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.217+0000 7f0265301640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 0x7f0244068990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f025c0af000 tx=0x7f025c0046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:42.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.221+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0258054210 con 0x7f02600c4990 2023-10-30T12:45:42.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f026015dcc0 con 0x7f02600c4990 2023-10-30T12:45:42.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0255ffb640 1 -- 172.21.15.8:0/1876594837 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f026015dcc0 con 0x7f02600c4990 2023-10-30T12:45:42.415 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-30T12:45:42.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 msgr2=0x7f0244068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 0x7f0244068990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f025c0af000 tx=0x7f025c0046d0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 msgr2=0x7f02600c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:42.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0258000f90 tx=0x7f0258011c20 comp rx=0 tx=0).stop 2023-10-30T12:45:42.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 shutdown_connections 2023-10-30T12:45:42.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f02440664d0 0x7f0244068990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02600c4990 0x7f02600c4e40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02600b3970 0x7f02600c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 --2- 172.21.15.8:0/1876594837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f024c003980 0x7f02601f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:42.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 >> 172.21.15.8:0/1876594837 conn(0x7f02600cef00 msgr2=0x7f02600c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:42.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 shutdown_connections 2023-10-30T12:45:42.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:42.413+0000 7f0266303640 1 -- 172.21.15.8:0/1876594837 wait complete. 2023-10-30T12:45:42.436 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-30T12:45:42.436 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-30T12:45:42.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2023-10-30T12:45:42.437 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-30T12:45:42.437 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:42.437 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-30T12:45:42.437 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-30T12:45:42.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-30T12:45:42.438 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-30T12:45:42.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.438 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2023-10-30T12:45:42.448 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-30T12:45:42.448 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-30T12:45:42.448 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2023-10-30T12:45:42.449 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2023-10-30T12:45:42.450 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:45:42.450 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2023-10-30T12:45:42.450 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2023-10-30T12:45:42.450 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-30T12:45:42.450 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 42398sec preferred_lft 42398sec 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:45:42.451 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 02:42:b6:cf:9e:b4 brd ff:ff:ff:ff:ff:ff 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-30T12:45:42.452 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-10-30T12:45:42.453 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-30T12:45:42.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.454 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-10-30T12:45:42.454 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-10-30T12:45:42.455 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2023-10-30T12:45:42.455 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2023-10-30T12:45:42.455 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2023-10-30T12:45:42.456 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-30T12:45:42.456 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-10-30T12:45:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.570 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-30T12:45:42.590 INFO:teuthology.orchestra.run.smithi008.stdout:1 2023-10-30T12:45:42.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.592 DEBUG:teuthology.orchestra.run.smithi008:> route 2023-10-30T12:45:42.661 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2023-10-30T12:45:42.661 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-30T12:45:42.661 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-30T12:45:42.661 INFO:teuthology.orchestra.run.smithi008.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-30T12:45:42.662 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-30T12:45:42.662 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-30T12:45:42.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.663 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-10-30T12:45:42.663 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-10-30T12:45:42.663 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-30T12:45:42.663 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-30T12:45:42.664 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-30T12:45:42.664 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-10-30T12:45:42.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.758 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-10-30T12:45:42.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.769 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2023-10-30T12:45:42.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.823 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-10-30T12:45:42.823 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-10-30T12:45:42.823 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-30T12:45:42.824 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-30T12:45:42.824 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-10-30T12:45:42.907 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-30T12:45:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:42.907 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-10-30T12:45:42.908 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-10-30T12:45:42.908 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-30T12:45:42.908 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-30T12:45:42.908 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-30T12:45:42.909 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-30T12:45:42.909 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-30T12:45:42.909 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-10-30T12:45:43.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:43.077 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-10-30T12:45:43.077 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-10-30T12:45:43.077 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2023-10-30T12:45:43.078 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2023-10-30T12:45:43.078 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-10-30T12:45:43.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-30T12:45:43.122 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:43.178 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-30T12:45:43.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-30T12:45:43.179 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:43.234 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-30T12:45:43.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:43.235 DEBUG:teuthology.orchestra.run.smithi008:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:45:43.340 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:45:43.340 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:45:43.340 INFO:teuthology.orchestra.run.smithi008.stdout:invalid new device string format 2023-10-30T12:45:43.452 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:45:43.453 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:45:43.453 INFO:teuthology.orchestra.run.smithi008.stdout:invalid new device string format 2023-10-30T12:45:43.453 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: resolved to: "172.21.15.8:3300,172.21.15.46:3300,172.21.15.46:3301" 2023-10-30T12:45:43.453 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-30T12:45:43.468 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-30T12:45:43.468 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-30T12:45:43.468 INFO:teuthology.orchestra.run.smithi008.stdout:invalid new device string format 2023-10-30T12:45:43.468 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: resolved to: "172.21.15.8:3300,172.21.15.46:3300,172.21.15.46:3301" 2023-10-30T12:45:43.469 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-30T12:45:43.469 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: resolved to: "172.21.15.8:6789,172.21.15.46:6789,172.21.15.46:6790" 2023-10-30T12:45:43.469 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.8:6789,172.21.15.46:6789,172.21.15.46:6790:/ 2023-10-30T12:45:43.469 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d8d9e729-1294-467e-ac5d-e55aafd29b99" will pass to kernel 2023-10-30T12:45:43.471 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-30T12:45:43.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:43.472 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:43.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:43.500 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:45:43.926 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:43.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:45:44.480 INFO:teuthology.orchestra.run.smithi008.stdout:client4768 2023-10-30T12:45:44.481 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:45:44.481 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:45:44.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:44.491 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:45:44.550 INFO:teuthology.orchestra.run.smithi008.stdout:client4768 2023-10-30T12:45:44.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:44.551 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-30T12:45:44.606 INFO:teuthology.orchestra.run.smithi008.stdout:client4768 2023-10-30T12:45:44.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:45:44.607 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d8d9e729-1294-467e-ac5d-e55aafd29b99.client4768/status') 2023-10-30T12:45:44.677 INFO:teuthology.orchestra.run.smithi008.stdout:instance: client.4768 (3)172.21.15.8:0/2038409912 2023-10-30T12:45:44.678 INFO:teuthology.orchestra.run.smithi008.stdout:blocklisted: false 2023-10-30T12:45:44.678 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records in 2023-10-30T12:45:44.678 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records out 2023-10-30T12:45:44.678 INFO:teuthology.orchestra.run.smithi008.stderr:69 bytes copied, 6.2989e-05 s, 1.1 MB/s 2023-10-30T12:45:44.680 INFO:teuthology.run_tasks:Running task mds_thrash... 2023-10-30T12:45:44.691 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:44.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 Processor -- start 2023-10-30T12:45:44.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 -- start start 2023-10-30T12:45:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 0x7f41f00c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41f00c3e40 0x7f41f0070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f00c2310 con 0x7f41f0150830 2023-10-30T12:45:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f006bc10 con 0x7f41f00c3e40 2023-10-30T12:45:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f746b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0156710 con 0x7f41f00cb7d0 2023-10-30T12:45:44.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f5a08640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41f00c3e40 0x7f41f0070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f4a06640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 0x7f41f00c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f5207640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f41f0150830 0x7f41f006b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:34532/0 (socket says 172.21.15.8:34532) 2023-10-30T12:45:44.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.869+0000 7f41f5207640 1 -- 172.21.15.8:0/2075702477 learned_addr learned my addr 172.21.15.8:0/2075702477 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41e7fff640 1 -- 172.21.15.8:0/2075702477 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855412146 0 0) 0x7f41f00c2310 con 0x7f41f0150830 2023-10-30T12:45:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41e7fff640 1 -- 172.21.15.8:0/2075702477 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41dc003530 con 0x7f41f0150830 2023-10-30T12:45:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f5a08640 1 -- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 msgr2=0x7f41f00c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f5a08640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 0x7f41f00c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f5a08640 1 -- 172.21.15.8:0/2075702477 >> v1:172.21.15.46:6789/0 conn(0x7f41f0150830 legacy=0x7f41f006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:44.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f5a08640 1 -- 172.21.15.8:0/2075702477 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f00c4360 con 0x7f41f00c3e40 2023-10-30T12:45:44.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f4a06640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 0x7f41f00c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:45:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.873+0000 7f41f5a08640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00c3e40 0x7f41f0070520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61a05f294942b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41e7fff640 1 -- 172.21.15.8:0/2075702477 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80af600 con 0x7f41f00c3e40 2023-10-30T12:45:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41e7fff640 1 -- 172.21.15.8:0/2075702477 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e80ab050 con 0x7f41f00c3e40 2023-10-30T12:45:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41e7fff640 1 -- 172.21.15.8:0/2075702477 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80af980 con 0x7f41f00c3e40 2023-10-30T12:45:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00c3e40 msgr2=0x7f41f0070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00c3e40 0x7f41f0070520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e801f130 comp rx=0 tx=0).stop 2023-10-30T12:45:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- 172.21.15.8:0/2075702477 shutdown_connections 2023-10-30T12:45:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00c3e40 0x7f41f0070520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- 172.21.15.8:0/2075702477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00cb7d0 0x7f41f00c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- 172.21.15.8:0/2075702477 >> 172.21.15.8:0/2075702477 conn(0x7f41f00cac70 msgr2=0x7f41f00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- 172.21.15.8:0/2075702477 shutdown_connections 2023-10-30T12:45:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- 172.21.15.8:0/2075702477 wait complete. 2023-10-30T12:45:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 Processor -- start 2023-10-30T12:45:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- start start 2023-10-30T12:45:44.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00c3e40 0x7f41f00c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 0x7f41f0155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0159230 con 0x7f41f0150830 2023-10-30T12:45:44.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0155720 con 0x7f41f00cb7d0 2023-10-30T12:45:44.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f746b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0157d90 con 0x7f41f00c3e40 2023-10-30T12:45:44.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f5a08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f5a08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53342/0 (socket says 172.21.15.8:53342) 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.877+0000 7f41f5a08640 1 -- 172.21.15.8:0/2965635436 learned_addr learned my addr 172.21.15.8:0/2965635436 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f4a06640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 0x7f41f0155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5207640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00c3e40 0x7f41f00c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 -- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00c3e40 msgr2=0x7f41f00c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00c3e40 0x7f41f00c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 -- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 msgr2=0x7f41f0155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 0x7f41f0155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 -- 172.21.15.8:0/2965635436 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009420 con 0x7f41f0150830 2023-10-30T12:45:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f4a06640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 0x7f41f0155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f5a08640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41e80b2040 tx=0x7f41e801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8009450 con 0x7f41f0150830 2023-10-30T12:45:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e80b9410 con 0x7f41f0150830 2023-10-30T12:45:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8009a70 con 0x7f41f0150830 2023-10-30T12:45:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80b97c0 con 0x7f41f0150830 2023-10-30T12:45:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f00ca6f0 con 0x7f41f0150830 2023-10-30T12:45:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f0066510 con 0x7f41f0150830 2023-10-30T12:45:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f41e80b6050 con 0x7f41f0150830 2023-10-30T12:45:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.881+0000 7f41e5ffb640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 0x7f41d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.885+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41e81223e0 con 0x7f41f0150830 2023-10-30T12:45:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.885+0000 7f41f5207640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 0x7f41d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.885+0000 7f41f5207640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 0x7f41d4068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0011f0 tx=0x7f41ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:44.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:44.885+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f41e80ef730 con 0x7f41f0150830 2023-10-30T12:45:45.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.045+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41f0155890 con 0x7f41f0150830 2023-10-30T12:45:45.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.045+0000 7f41e5ffb640 1 -- 172.21.15.8:0/2965635436 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f41e80b4090 con 0x7f41f0150830 2023-10-30T12:45:45.048 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:45.049 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 msgr2=0x7f41d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 0x7f41d4068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0011f0 tx=0x7f41ec009040 comp rx=0 tx=0).stop 2023-10-30T12:45:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 msgr2=0x7f41f01484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41e80b2040 tx=0x7f41e801bed0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 shutdown_connections 2023-10-30T12:45:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f41d40664a0 0x7f41d4068960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00cb7d0 0x7f41f0155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41f0150830 0x7f41f01484e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 --2- 172.21.15.8:0/2965635436 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f41f00c3e40 0x7f41f00c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 >> 172.21.15.8:0/2965635436 conn(0x7f41f00cac70 msgr2=0x7f41f006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 shutdown_connections 2023-10-30T12:45:45.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.049+0000 7f41f746b640 1 -- 172.21.15.8:0/2965635436 wait complete. 2023-10-30T12:45:45.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:45.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 Processor -- start 2023-10-30T12:45:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 -- start start 2023-10-30T12:45:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc401611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc4015e650 0x7fbc400c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc400717a0 con 0x7fbc400c0840 2023-10-30T12:45:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc40162170 con 0x7fbc4015e650 2023-10-30T12:45:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc47a17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401616f0 con 0x7fbc40158070 2023-10-30T12:45:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc457b3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc401611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc44fb2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc4015e650 0x7fbc400c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc45fb4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc400c0840 0x7fbc40071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:34548/0 (socket says 172.21.15.8:34548) 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.213+0000 7fbc45fb4640 1 -- 172.21.15.8:0/2524829624 learned_addr learned my addr 172.21.15.8:0/2524829624 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc37fff640 1 -- 172.21.15.8:0/2524829624 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605795880 0 0) 0x7fbc400717a0 con 0x7fbc400c0840 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc37fff640 1 -- 172.21.15.8:0/2524829624 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2c003530 con 0x7fbc400c0840 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc457b3640 1 -- 172.21.15.8:0/2524829624 >> v2:172.21.15.46:3300/0 conn(0x7fbc4015e650 msgr2=0x7fbc400c0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc457b3640 1 --2- 172.21.15.8:0/2524829624 >> v2:172.21.15.46:3300/0 conn(0x7fbc4015e650 0x7fbc400c0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc457b3640 1 -- 172.21.15.8:0/2524829624 >> v1:172.21.15.46:6789/0 conn(0x7fbc400c0840 legacy=0x7fbc40071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc457b3640 1 -- 172.21.15.8:0/2524829624 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc401618d0 con 0x7fbc40158070 2023-10-30T12:45:45.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc457b3640 1 --2- 172.21.15.8:0/2524829624 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc401611b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0097c0 tx=0x7fbc3c01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51d461bb5590d01 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc37fff640 1 -- 172.21.15.8:0/2524829624 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0a8970 con 0x7fbc40158070 2023-10-30T12:45:45.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc37fff640 1 -- 172.21.15.8:0/2524829624 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0a8ad0 con 0x7fbc40158070 2023-10-30T12:45:45.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.217+0000 7fbc37fff640 1 -- 172.21.15.8:0/2524829624 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0a8e60 con 0x7fbc40158070 2023-10-30T12:45:45.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- 172.21.15.8:0/2524829624 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 msgr2=0x7fbc401611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- 172.21.15.8:0/2524829624 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc401611b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0097c0 tx=0x7fbc3c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:45.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- 172.21.15.8:0/2524829624 shutdown_connections 2023-10-30T12:45:45.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- 172.21.15.8:0/2524829624 >> v2:172.21.15.46:3300/0 conn(0x7fbc4015e650 0x7fbc400c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- 172.21.15.8:0/2524829624 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc401611b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- 172.21.15.8:0/2524829624 >> 172.21.15.8:0/2524829624 conn(0x7fbc4006da40 msgr2=0x7fbc4006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- 172.21.15.8:0/2524829624 shutdown_connections 2023-10-30T12:45:45.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- 172.21.15.8:0/2524829624 wait complete. 2023-10-30T12:45:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 Processor -- start 2023-10-30T12:45:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- start start 2023-10-30T12:45:45.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 0x7fbc400c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc4015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401f3320 con 0x7fbc4015e650 2023-10-30T12:45:45.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401f29a0 con 0x7fbc400c0840 2023-10-30T12:45:45.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc47a17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400c2340 con 0x7fbc40158070 2023-10-30T12:45:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53344/0 (socket says 172.21.15.8:53344) 2023-10-30T12:45:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 -- 172.21.15.8:0/420917041 learned_addr learned my addr 172.21.15.8:0/420917041 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:45.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc457b3640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 0x7fbc400c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc44fb2640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc4015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 -- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 msgr2=0x7fbc4015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc4015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 -- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 msgr2=0x7fbc400c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 0x7fbc400c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc45fb4640 1 -- 172.21.15.8:0/420917041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c009420 con 0x7fbc4015e650 2023-10-30T12:45:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.221+0000 7fbc457b3640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 0x7fbc400c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc45fb4640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cb40 tx=0x7fbc3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800aca0 con 0x7fbc4015e650 2023-10-30T12:45:45.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800ae00 con 0x7fbc4015e650 2023-10-30T12:45:45.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc400c2610 con 0x7fbc4015e650 2023-10-30T12:45:45.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38012620 con 0x7fbc4015e650 2023-10-30T12:45:45.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc401f2c60 con 0x7fbc4015e650 2023-10-30T12:45:45.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc401f3460 con 0x7fbc4015e650 2023-10-30T12:45:45.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc38007070 con 0x7fbc4015e650 2023-10-30T12:45:45.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 0x7fbc24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc38083e70 con 0x7fbc4015e650 2023-10-30T12:45:45.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.225+0000 7fbc457b3640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 0x7fbc24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.229+0000 7fbc457b3640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 0x7fbc24068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0b2000 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.229+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc401f3460 con 0x7fbc4015e650 2023-10-30T12:45:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.385+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc401f3460 con 0x7fbc4015e650 2023-10-30T12:45:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc35ffb640 1 -- 172.21.15.8:0/420917041 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc401f3460 con 0x7fbc4015e650 2023-10-30T12:45:45.391 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 msgr2=0x7fbc24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 0x7fbc24068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0b2000 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2023-10-30T12:45:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 msgr2=0x7fbc4014fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cb40 tx=0x7fbc3800a060 comp rx=0 tx=0).stop 2023-10-30T12:45:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 shutdown_connections 2023-10-30T12:45:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fbc240664a0 0x7fbc24068960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc4015e650 0x7fbc4014fa40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fbc40158070 0x7fbc4015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 --2- 172.21.15.8:0/420917041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc400c0840 0x7fbc400c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 >> 172.21.15.8:0/420917041 conn(0x7fbc4006da40 msgr2=0x7fbc4015caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 shutdown_connections 2023-10-30T12:45:45.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.389+0000 7fbc47a17640 1 -- 172.21.15.8:0/420917041 wait complete. 2023-10-30T12:45:45.412 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:45:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.553+0000 7f026ee67640 1 Processor -- start 2023-10-30T12:45:45.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.553+0000 7f026ee67640 1 -- start start 2023-10-30T12:45:45.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ee67640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 0x7f02680c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ee67640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0268071cb0 0x7f026815a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ee67640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026815a770 con 0x7f0268153300 2023-10-30T12:45:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ee67640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680cca10 con 0x7f0268071cb0 2023-10-30T12:45:45.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ee67640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026815d0a0 con 0x7f02680718d0 2023-10-30T12:45:45.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026d664640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0268071cb0 0x7f026815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026de65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 0x7f02680c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026e666640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0268153300 0x7f02680cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:34560/0 (socket says 172.21.15.8:34560) 2023-10-30T12:45:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026e666640 1 -- 172.21.15.8:0/1004623522 learned_addr learned my addr 172.21.15.8:0/1004623522 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ce63640 1 -- 172.21.15.8:0/1004623522 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079599373 0 0) 0x7f026815a770 con 0x7f0268153300 2023-10-30T12:45:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026ce63640 1 -- 172.21.15.8:0/1004623522 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0254003530 con 0x7f0268153300 2023-10-30T12:45:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026de65640 1 -- 172.21.15.8:0/1004623522 >> v2:172.21.15.46:3300/0 conn(0x7f0268071cb0 msgr2=0x7f026815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026de65640 1 --2- 172.21.15.8:0/1004623522 >> v2:172.21.15.46:3300/0 conn(0x7f0268071cb0 0x7f026815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026de65640 1 -- 172.21.15.8:0/1004623522 >> v1:172.21.15.46:6789/0 conn(0x7f0268153300 legacy=0x7f02680cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.557+0000 7f026de65640 1 -- 172.21.15.8:0/1004623522 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026815d380 con 0x7f02680718d0 2023-10-30T12:45:45.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026de65640 1 --2- 172.21.15.8:0/1004623522 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 0x7f02680c9a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02640097c0 tx=0x7f026401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5fde08e0558de8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ce63640 1 -- 172.21.15.8:0/1004623522 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640a8970 con 0x7f02680718d0 2023-10-30T12:45:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ce63640 1 -- 172.21.15.8:0/1004623522 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640a8ad0 con 0x7f02680718d0 2023-10-30T12:45:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ce63640 1 -- 172.21.15.8:0/1004623522 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640a8e60 con 0x7f02680718d0 2023-10-30T12:45:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 -- 172.21.15.8:0/1004623522 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 msgr2=0x7f02680c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 --2- 172.21.15.8:0/1004623522 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 0x7f02680c9a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02640097c0 tx=0x7f026401ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 -- 172.21.15.8:0/1004623522 shutdown_connections 2023-10-30T12:45:45.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 --2- 172.21.15.8:0/1004623522 >> v2:172.21.15.46:3300/0 conn(0x7f0268071cb0 0x7f026815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 --2- 172.21.15.8:0/1004623522 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f02680718d0 0x7f02680c9a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 -- 172.21.15.8:0/1004623522 >> 172.21.15.8:0/1004623522 conn(0x7f026806ab00 msgr2=0x7f0268065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 -- 172.21.15.8:0/1004623522 shutdown_connections 2023-10-30T12:45:45.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 -- 172.21.15.8:0/1004623522 wait complete. 2023-10-30T12:45:45.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.561+0000 7f026ee67640 1 Processor -- start 2023-10-30T12:45:45.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- start start 2023-10-30T12:45:45.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 0x7f026815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0268153300 0x7f026815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268158670 con 0x7f02680718d0 2023-10-30T12:45:45.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268159590 con 0x7f0268071cb0 2023-10-30T12:45:45.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0268158860 con 0x7f0268153300 2023-10-30T12:45:45.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53348/0 (socket says 172.21.15.8:53348) 2023-10-30T12:45:45.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 -- 172.21.15.8:0/2630820267 learned_addr learned my addr 172.21.15.8:0/2630820267 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:45.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026d664640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 0x7f026815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026e666640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0268153300 0x7f026815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 -- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0268153300 msgr2=0x7f026815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0268153300 0x7f026815c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 -- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 msgr2=0x7f026815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 0x7f026815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 -- 172.21.15.8:0/2630820267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0264009420 con 0x7f02680718d0 2023-10-30T12:45:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026d664640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 0x7f026815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:45.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026de65640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02640b3040 tx=0x7f026401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640b0d60 con 0x7f02680718d0 2023-10-30T12:45:45.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640b9430 con 0x7f02680718d0 2023-10-30T12:45:45.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0268159830 con 0x7f02680718d0 2023-10-30T12:45:45.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640b97e0 con 0x7f02680718d0 2023-10-30T12:45:45.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02680c7580 con 0x7f02680718d0 2023-10-30T12:45:45.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.565+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680c7770 con 0x7f02680718d0 2023-10-30T12:45:45.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.569+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f02640b7040 con 0x7f02680718d0 2023-10-30T12:45:45.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.569+0000 7f025e7fc640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 0x7f024c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.569+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0264121a60 con 0x7f02680718d0 2023-10-30T12:45:45.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.569+0000 7f026d664640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 0x7f024c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.569+0000 7f026d664640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 0x7f024c068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0268157370 tx=0x7f0258008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.573+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f02640eedb0 con 0x7f02680718d0 2023-10-30T12:45:45.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.729+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f026804e170 con 0x7f02680718d0 2023-10-30T12:45:45.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.729+0000 7f025e7fc640 1 -- 172.21.15.8:0/2630820267 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02640b5020 con 0x7f02680718d0 2023-10-30T12:45:45.733 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:45:45.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 msgr2=0x7f024c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 0x7f024c068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0268157370 tx=0x7f0258008040 comp rx=0 tx=0).stop 2023-10-30T12:45:45.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 msgr2=0x7f02680c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02640b3040 tx=0x7f026401ba10 comp rx=0 tx=0).stop 2023-10-30T12:45:45.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 shutdown_connections 2023-10-30T12:45:45.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f024c0664a0 0x7f024c068960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0268071cb0 0x7f026815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0268153300 0x7f026815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 --2- 172.21.15.8:0/2630820267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02680718d0 0x7f02680c6a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 >> 172.21.15.8:0/2630820267 conn(0x7f026806ab00 msgr2=0x7f0268151980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 shutdown_connections 2023-10-30T12:45:45.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.733+0000 7f026ee67640 1 -- 172.21.15.8:0/2630820267 wait complete. 2023-10-30T12:45:45.753 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:45:45.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 Processor -- start 2023-10-30T12:45:45.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 -- start start 2023-10-30T12:45:45.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c00cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28c00c4810 0x7f28c015a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c0153f80 con 0x7f28c00c4410 2023-10-30T12:45:45.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c0159df0 con 0x7f28c00c4810 2023-10-30T12:45:45.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c7301640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015a7c0 con 0x7f28c0153ba0 2023-10-30T12:45:45.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c489c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f28c00c4410 0x7f28c015cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60458/0 (socket says 172.21.15.8:60458) 2023-10-30T12:45:45.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c489c640 1 -- 172.21.15.8:0/993470752 learned_addr learned my addr 172.21.15.8:0/993470752 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:45.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c509d640 1 --2- 172.21.15.8:0/993470752 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c00cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c589e640 1 --2- 172.21.15.8:0/993470752 >> v2:172.21.15.46:3300/0 conn(0x7f28c00c4810 0x7f28c015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344679019 0 0) 0x7f28c0153f80 con 0x7f28c00c4410 2023-10-30T12:45:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003530 con 0x7f28c00c4410 2023-10-30T12:45:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737110960 0 0) 0x7f28ac003530 con 0x7f28c00c4410 2023-10-30T12:45:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c509d640 1 -- 172.21.15.8:0/993470752 >> v2:172.21.15.46:3300/0 conn(0x7f28c00c4810 msgr2=0x7f28c015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c509d640 1 --2- 172.21.15.8:0/993470752 >> v2:172.21.15.46:3300/0 conn(0x7f28c00c4810 0x7f28c015a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c509d640 1 -- 172.21.15.8:0/993470752 >> v1:172.21.15.8:6789/0 conn(0x7f28c00c4410 legacy=0x7f28c015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.897+0000 7f28c509d640 1 -- 172.21.15.8:0/993470752 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c01597c0 con 0x7f28c0153ba0 2023-10-30T12:45:45.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c509d640 1 --2- 172.21.15.8:0/993470752 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c00cb4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0097c0 tx=0x7f28bc01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07ef50c7c1afd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0001680 con 0x7f28c0153ba0 2023-10-30T12:45:45.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a7050 con 0x7f28c0153ba0 2023-10-30T12:45:45.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28b7fff640 1 -- 172.21.15.8:0/993470752 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0ab600 con 0x7f28c0153ba0 2023-10-30T12:45:45.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 -- 172.21.15.8:0/993470752 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 msgr2=0x7f28c00cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 --2- 172.21.15.8:0/993470752 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c00cb4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0097c0 tx=0x7f28bc01c2e0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 -- 172.21.15.8:0/993470752 shutdown_connections 2023-10-30T12:45:45.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 --2- 172.21.15.8:0/993470752 >> v2:172.21.15.46:3300/0 conn(0x7f28c00c4810 0x7f28c015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 --2- 172.21.15.8:0/993470752 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c00cb4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 -- 172.21.15.8:0/993470752 >> 172.21.15.8:0/993470752 conn(0x7f28c0071790 msgr2=0x7f28c0071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:45.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 -- 172.21.15.8:0/993470752 shutdown_connections 2023-10-30T12:45:45.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.901+0000 7f28c7301640 1 -- 172.21.15.8:0/993470752 wait complete. 2023-10-30T12:45:45.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 Processor -- start 2023-10-30T12:45:45.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- start start 2023-10-30T12:45:45.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00c4410 0x7f28c015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c01f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00c6df0 con 0x7f28c00c4810 2023-10-30T12:45:45.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00b4960 con 0x7f28c00c4410 2023-10-30T12:45:45.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00c7210 con 0x7f28c0153ba0 2023-10-30T12:45:45.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53356/0 (socket says 172.21.15.8:53356) 2023-10-30T12:45:45.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 -- 172.21.15.8:0/2410915519 learned_addr learned my addr 172.21.15.8:0/2410915519 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:45.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c589e640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c01f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c509d640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00c4410 0x7f28c015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 -- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 msgr2=0x7f28c01f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c01f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 -- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00c4410 msgr2=0x7f28c015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:45.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00c4410 0x7f28c015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:45.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 -- 172.21.15.8:0/2410915519 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009420 con 0x7f28c00c4810 2023-10-30T12:45:45.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c489c640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c780 tx=0x7f28b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0005ae0 con 0x7f28c00c4810 2023-10-30T12:45:45.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0005c40 con 0x7f28c00c4810 2023-10-30T12:45:45.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0007070 con 0x7f28c00c4810 2023-10-30T12:45:45.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c00c6450 con 0x7f28c00c4810 2023-10-30T12:45:45.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.905+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c00c6a10 con 0x7f28c00c4810 2023-10-30T12:45:45.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c00c6f30 con 0x7f28c00c4810 2023-10-30T12:45:45.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28b00071f0 con 0x7f28c00c4810 2023-10-30T12:45:45.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28b5ffb640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:45.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28b0083dc0 con 0x7f28c00c4810 2023-10-30T12:45:45.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28c509d640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:45.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.909+0000 7f28c509d640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 0x7f28a4068990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0ad000 tx=0x7f28bc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:45.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:45.913+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28c00c6f30 con 0x7f28c00c4810 2023-10-30T12:45:46.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28c00c6f30 con 0x7f28c00c4810 2023-10-30T12:45:46.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28b5ffb640 1 -- 172.21.15.8:0/2410915519 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28c00c6f30 con 0x7f28c00c4810 2023-10-30T12:45:46.072 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:45:46.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 msgr2=0x7f28a4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 0x7f28a4068990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0ad000 tx=0x7f28bc0a6000 comp rx=0 tx=0).stop 2023-10-30T12:45:46.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 msgr2=0x7f28c0152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c780 tx=0x7f28b000cc40 comp rx=0 tx=0).stop 2023-10-30T12:45:46.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 shutdown_connections 2023-10-30T12:45:46.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f28a40664d0 0x7f28a4068990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f28c0153ba0 0x7f28c01f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28c00c4810 0x7f28c0152330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 --2- 172.21.15.8:0/2410915519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00c4410 0x7f28c015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 >> 172.21.15.8:0/2410915519 conn(0x7f28c0071790 msgr2=0x7f28c0157490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 shutdown_connections 2023-10-30T12:45:46.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.069+0000 7f28c7301640 1 -- 172.21.15.8:0/2410915519 wait complete. 2023-10-30T12:45:46.093 INFO:tasks.mds_thrash:mds thrasher using random seed: 1698669946 2023-10-30T12:45:46.094 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:46.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 Processor -- start 2023-10-30T12:45:46.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 -- start start 2023-10-30T12:45:46.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25d0153c80 0x7f25d015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0159f20 con 0x7f25d00c42a0 2023-10-30T12:45:46.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d015d620 con 0x7f25d00c4a10 2023-10-30T12:45:46.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25d4b3e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d015dec0 con 0x7f25d0153c80 2023-10-30T12:45:46.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ced9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f25d00c42a0 0x7f25d00ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60472/0 (socket says 172.21.15.8:60472) 2023-10-30T12:45:46.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ced9d640 1 -- 172.21.15.8:0/41038587 learned_addr learned my addr 172.21.15.8:0/41038587 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:46.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ce59c640 1 --2- 172.21.15.8:0/41038587 >> v2:172.21.15.46:3300/0 conn(0x7f25d0153c80 0x7f25d015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902004409 0 0) 0x7f25d0159f20 con 0x7f25d00c42a0 2023-10-30T12:45:46.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25bc003530 con 0x7f25d00c42a0 2023-10-30T12:45:46.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129991741 0 0) 0x7f25bc003530 con 0x7f25d00c42a0 2023-10-30T12:45:46.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ce59c640 1 -- 172.21.15.8:0/41038587 >> v1:172.21.15.46:6789/0 conn(0x7f25d00c4a10 legacy=0x7f25d015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ce59c640 1 -- 172.21.15.8:0/41038587 >> v1:172.21.15.8:6789/0 conn(0x7f25d00c42a0 legacy=0x7f25d00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.237+0000 7f25ce59c640 1 -- 172.21.15.8:0/41038587 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d015d900 con 0x7f25d0153c80 2023-10-30T12:45:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.241+0000 7f25ce59c640 1 --2- 172.21.15.8:0/41038587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d0153c80 0x7f25d015cf90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097c0 tx=0x7f25c001eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2e0eb740dbfd61 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.245+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8001a00 con 0x7f25d0153c80 2023-10-30T12:45:46.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.245+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c00a8610 con 0x7f25d0153c80 2023-10-30T12:45:46.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.245+0000 7f25cdd9b640 1 -- 172.21.15.8:0/41038587 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00a89a0 con 0x7f25d0153c80 2023-10-30T12:45:46.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- 172.21.15.8:0/41038587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d0153c80 msgr2=0x7f25d015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/41038587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d0153c80 0x7f25d015cf90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097c0 tx=0x7f25c001eb90 comp rx=0 tx=0).stop 2023-10-30T12:45:46.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- 172.21.15.8:0/41038587 shutdown_connections 2023-10-30T12:45:46.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/41038587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d0153c80 0x7f25d015cf90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- 172.21.15.8:0/41038587 >> 172.21.15.8:0/41038587 conn(0x7f25d00715c0 msgr2=0x7f25d00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- 172.21.15.8:0/41038587 shutdown_connections 2023-10-30T12:45:46.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- 172.21.15.8:0/41038587 wait complete. 2023-10-30T12:45:46.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 Processor -- start 2023-10-30T12:45:46.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- start start 2023-10-30T12:45:46.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25d00c42a0 0x7f25d0159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d00c4a10 0x7f25d006c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01561f0 con 0x7f25d0153c80 2023-10-30T12:45:46.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0154e10 con 0x7f25d00c4a10 2023-10-30T12:45:46.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25d4b3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0158870 con 0x7f25d00c42a0 2023-10-30T12:45:46.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ce59c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25d00c42a0 0x7f25d0159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53358/0 (socket says 172.21.15.8:53358) 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 -- 172.21.15.8:0/2617636477 learned_addr learned my addr 172.21.15.8:0/2617636477 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25c7fff640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d00c4a10 0x7f25d006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 -- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25d00c42a0 msgr2=0x7f25d0159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25d00c42a0 0x7f25d0159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 -- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d00c4a10 msgr2=0x7f25d006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d00c4a10 0x7f25d006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.249+0000 7f25ced9d640 1 -- 172.21.15.8:0/2617636477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009420 con 0x7f25d0153c80 2023-10-30T12:45:46.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25ced9d640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f25c80018d0 tx=0x7f25c800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8012400 con 0x7f25d0153c80 2023-10-30T12:45:46.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8012560 con 0x7f25d0153c80 2023-10-30T12:45:46.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8012910 con 0x7f25d0153c80 2023-10-30T12:45:46.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d0155440 con 0x7f25d0153c80 2023-10-30T12:45:46.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0159990 con 0x7f25d0153c80 2023-10-30T12:45:46.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0156330 con 0x7f25d0153c80 2023-10-30T12:45:46.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25c8012a70 con 0x7f25d0153c80 2023-10-30T12:45:46.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 0x7f25b0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25c80845f0 con 0x7f25d0153c80 2023-10-30T12:45:46.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.253+0000 7f25ce59c640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 0x7f25b0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.257+0000 7f25ce59c640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 0x7f25b0068990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c00b2000 tx=0x7f25c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.257+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f25d0156330 con 0x7f25d0153c80 2023-10-30T12:45:46.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.421+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25d0156330 con 0x7f25d0153c80 2023-10-30T12:45:46.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.421+0000 7f25c6ffd640 1 -- 172.21.15.8:0/2617636477 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f25d0156330 con 0x7f25d0153c80 2023-10-30T12:45:46.425 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:46.425 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:46.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.421+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 msgr2=0x7f25b0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.421+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 0x7f25b0068990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c00b2000 tx=0x7f25c0008040 comp rx=0 tx=0).stop 2023-10-30T12:45:46.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 msgr2=0x7f25d0149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f25c80018d0 tx=0x7f25c800a430 comp rx=0 tx=0).stop 2023-10-30T12:45:46.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 shutdown_connections 2023-10-30T12:45:46.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25b00664d0 0x7f25b0068990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d00c4a10 0x7f25d006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25d0153c80 0x7f25d0149ba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 --2- 172.21.15.8:0/2617636477 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25d00c42a0 0x7f25d0159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 >> 172.21.15.8:0/2617636477 conn(0x7f25d00715c0 msgr2=0x7f25d0152410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 shutdown_connections 2023-10-30T12:45:46.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.425+0000 7f25d4b3e640 1 -- 172.21.15.8:0/2617636477 wait complete. 2023-10-30T12:45:46.446 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:46.446 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:46.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 Processor -- start 2023-10-30T12:45:46.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 -- start start 2023-10-30T12:45:46.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 0x7f6304072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63041581f0 con 0x7f63040725c0 2023-10-30T12:45:46.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6304151e00 con 0x7f6304151a20 2023-10-30T12:45:46.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f6309891640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63041576d0 con 0x7f63040c1020 2023-10-30T12:45:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f63037fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f63040725c0 0x7f63040c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60484/0 (socket says 172.21.15.8:60484) 2023-10-30T12:45:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f63037fe640 1 -- 172.21.15.8:0/2020616877 learned_addr learned my addr 172.21.15.8:0/2020616877 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.589+0000 7f62fbfff640 1 --2- 172.21.15.8:0/2020616877 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 0x7f6304072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310067343 0 0) 0x7f63041581f0 con 0x7f63040725c0 2023-10-30T12:45:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f0003530 con 0x7f63040725c0 2023-10-30T12:45:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968412329 0 0) 0x7f6304151e00 con 0x7f6304151a20 2023-10-30T12:45:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63041581f0 con 0x7f6304151a20 2023-10-30T12:45:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f62fbfff640 1 -- 172.21.15.8:0/2020616877 >> v1:172.21.15.46:6789/0 conn(0x7f6304151a20 legacy=0x7f63040cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f62fbfff640 1 -- 172.21.15.8:0/2020616877 >> v1:172.21.15.8:6789/0 conn(0x7f63040725c0 legacy=0x7f63040c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784972493 0 0) 0x7f62f0003530 con 0x7f63040725c0 2023-10-30T12:45:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f62fbfff640 1 -- 172.21.15.8:0/2020616877 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040c19d0 con 0x7f63040c1020 2023-10-30T12:45:46.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f62fbfff640 1 --2- 172.21.15.8:0/2020616877 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 0x7f6304072080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0097f0 tx=0x7f62ec01ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8cfc0d61bbdfa35 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc001a00 con 0x7f63040c1020 2023-10-30T12:45:46.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.593+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec0a8650 con 0x7f63040c1020 2023-10-30T12:45:46.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f63027fc640 1 -- 172.21.15.8:0/2020616877 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a89c0 con 0x7f63040c1020 2023-10-30T12:45:46.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- 172.21.15.8:0/2020616877 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 msgr2=0x7f6304072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 --2- 172.21.15.8:0/2020616877 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 0x7f6304072080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0097f0 tx=0x7f62ec01ebd0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- 172.21.15.8:0/2020616877 shutdown_connections 2023-10-30T12:45:46.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 --2- 172.21.15.8:0/2020616877 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040c1020 0x7f6304072080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- 172.21.15.8:0/2020616877 >> 172.21.15.8:0/2020616877 conn(0x7f630406da40 msgr2=0x7f630406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- 172.21.15.8:0/2020616877 shutdown_connections 2023-10-30T12:45:46.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- 172.21.15.8:0/2020616877 wait complete. 2023-10-30T12:45:46.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 Processor -- start 2023-10-30T12:45:46.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- start start 2023-10-30T12:45:46.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040725c0 0x7f63041522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 0x7f6304071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6304154a90 con 0x7f63040c1020 2023-10-30T12:45:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6304071500 con 0x7f6304151a20 2023-10-30T12:45:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6309891640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630406fe70 con 0x7f63040725c0 2023-10-30T12:45:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53374/0 (socket says 172.21.15.8:53374) 2023-10-30T12:45:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f62fbfff640 1 -- 172.21.15.8:0/2386908456 learned_addr learned my addr 172.21.15.8:0/2386908456 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:46.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f6302ffd640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040725c0 0x7f63041522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f63037fe640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 0x7f6304071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.597+0000 7f62fbfff640 1 -- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040725c0 msgr2=0x7f63041522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f62fbfff640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040725c0 0x7f63041522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f62fbfff640 1 -- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 msgr2=0x7f6304071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f62fbfff640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 0x7f6304071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f62fbfff640 1 -- 172.21.15.8:0/2386908456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec009420 con 0x7f63040c1020 2023-10-30T12:45:46.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f63037fe640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 0x7f6304071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:46.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f62fbfff640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c10 tx=0x7f62ec01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0bd070 con 0x7f63040c1020 2023-10-30T12:45:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec0a8a90 con 0x7f63040c1020 2023-10-30T12:45:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a8e00 con 0x7f63040c1020 2023-10-30T12:45:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304153c90 con 0x7f63040c1020 2023-10-30T12:45:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304154fa0 con 0x7f63040c1020 2023-10-30T12:45:46.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63040b1250 con 0x7f63040c1020 2023-10-30T12:45:46.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f62ec0b7040 con 0x7f63040c1020 2023-10-30T12:45:46.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 0x7f62e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.601+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62ec120bc0 con 0x7f63040c1020 2023-10-30T12:45:46.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.605+0000 7f6302ffd640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 0x7f62e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.605+0000 7f6302ffd640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 0x7f62e4068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62f40011f0 tx=0x7f62f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.605+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f62ec0ee030 con 0x7f63040c1020 2023-10-30T12:45:46.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.765+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f630404e170 con 0x7f63040c1020 2023-10-30T12:45:46.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.765+0000 7f6300ff9640 1 -- 172.21.15.8:0/2386908456 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62ec0f1650 con 0x7f63040c1020 2023-10-30T12:45:46.769 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 msgr2=0x7f62e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 0x7f62e4068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62f40011f0 tx=0x7f62f4009040 comp rx=0 tx=0).stop 2023-10-30T12:45:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 msgr2=0x7f6304071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c10 tx=0x7f62ec01b970 comp rx=0 tx=0).stop 2023-10-30T12:45:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 shutdown_connections 2023-10-30T12:45:46.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f62e40664a0 0x7f62e4068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304151a20 0x7f6304071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63040c1020 0x7f6304071990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 --2- 172.21.15.8:0/2386908456 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f63040725c0 0x7f63041522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 >> 172.21.15.8:0/2386908456 conn(0x7f630406da40 msgr2=0x7f6304156a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 shutdown_connections 2023-10-30T12:45:46.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.769+0000 7f6309891640 1 -- 172.21.15.8:0/2386908456 wait complete. 2023-10-30T12:45:46.791 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:45:46.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 Processor -- start 2023-10-30T12:45:46.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 -- start start 2023-10-30T12:45:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80700c1020 0x7f8070072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f80700725c0 0x7f80700c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80701581f0 con 0x7f8070151a20 2023-10-30T12:45:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80701576d0 con 0x7f80700725c0 2023-10-30T12:45:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8074b1d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8070158680 con 0x7f80700c1020 2023-10-30T12:45:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f806ed9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f80700725c0 0x7f80700c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f806e59c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8070151a20 0x7f80700cb4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60488/0 (socket says 172.21.15.8:60488) 2023-10-30T12:45:46.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f806e59c640 1 -- 172.21.15.8:0/1130642285 learned_addr learned my addr 172.21.15.8:0/1130642285 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.933+0000 7f8067fff640 1 --2- 172.21.15.8:0/1130642285 >> v2:172.21.15.46:3300/0 conn(0x7f80700c1020 0x7f8070072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806dd9b640 1 -- 172.21.15.8:0/1130642285 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344943491 0 0) 0x7f80701581f0 con 0x7f8070151a20 2023-10-30T12:45:46.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806dd9b640 1 -- 172.21.15.8:0/1130642285 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f805c003530 con 0x7f8070151a20 2023-10-30T12:45:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806ed9d640 1 -- 172.21.15.8:0/1130642285 >> v2:172.21.15.46:3300/0 conn(0x7f80700c1020 msgr2=0x7f8070072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806ed9d640 1 --2- 172.21.15.8:0/1130642285 >> v2:172.21.15.46:3300/0 conn(0x7f80700c1020 0x7f8070072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806ed9d640 1 -- 172.21.15.8:0/1130642285 >> v1:172.21.15.8:6789/0 conn(0x7f8070151a20 legacy=0x7f80700cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806ed9d640 1 -- 172.21.15.8:0/1130642285 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070151e00 con 0x7f80700725c0 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806ed9d640 1 --2- 172.21.15.8:0/1130642285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700725c0 0x7f80700c74a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80700cb440 tx=0x7f806801f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c5302126d212 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806dd9b640 1 -- 172.21.15.8:0/1130642285 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680af600 con 0x7f80700725c0 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806dd9b640 1 -- 172.21.15.8:0/1130642285 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80680ab050 con 0x7f80700725c0 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f806dd9b640 1 -- 172.21.15.8:0/1130642285 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680af980 con 0x7f80700725c0 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f8074b1d640 1 -- 172.21.15.8:0/1130642285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700725c0 msgr2=0x7f80700c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.937+0000 7f8074b1d640 1 --2- 172.21.15.8:0/1130642285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700725c0 0x7f80700c74a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80700cb440 tx=0x7f806801f100 comp rx=0 tx=0).stop 2023-10-30T12:45:46.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- 172.21.15.8:0/1130642285 shutdown_connections 2023-10-30T12:45:46.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 --2- 172.21.15.8:0/1130642285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700725c0 0x7f80700c74a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 --2- 172.21.15.8:0/1130642285 >> v2:172.21.15.46:3300/0 conn(0x7f80700c1020 0x7f8070072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- 172.21.15.8:0/1130642285 >> 172.21.15.8:0/1130642285 conn(0x7f807006da40 msgr2=0x7f807006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:46.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- 172.21.15.8:0/1130642285 shutdown_connections 2023-10-30T12:45:46.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- 172.21.15.8:0/1130642285 wait complete. 2023-10-30T12:45:46.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 Processor -- start 2023-10-30T12:45:46.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- start start 2023-10-30T12:45:46.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80700725c0 0x7f8070156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8070151a20 0x7f8070071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701571e0 con 0x7f80700c1020 2023-10-30T12:45:46.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c8c90 con 0x7f80700725c0 2023-10-30T12:45:46.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8074b1d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c9870 con 0x7f8070151a20 2023-10-30T12:45:46.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53392/0 (socket says 172.21.15.8:53392) 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 -- 172.21.15.8:0/3293258386 learned_addr learned my addr 172.21.15.8:0/3293258386 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f806ed9d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8070151a20 0x7f8070071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f806e59c640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80700725c0 0x7f8070156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 -- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8070151a20 msgr2=0x7f8070071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8070151a20 0x7f8070071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 -- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80700725c0 msgr2=0x7f8070156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:46.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80700725c0 0x7f8070156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:46.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 -- 172.21.15.8:0/3293258386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068009a70 con 0x7f80700c1020 2023-10-30T12:45:46.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.941+0000 7f8067fff640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f805800e870 tx=0x7f805800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058009510 con 0x7f80700c1020 2023-10-30T12:45:46.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805800ee60 con 0x7f80700c1020 2023-10-30T12:45:46.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800cae0 con 0x7f80700c1020 2023-10-30T12:45:46.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80700c4740 con 0x7f80700c1020 2023-10-30T12:45:46.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80700c9220 con 0x7f80700c1020 2023-10-30T12:45:46.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807004e170 con 0x7f80700c1020 2023-10-30T12:45:46.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f805800cc40 con 0x7f80700c1020 2023-10-30T12:45:46.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 0x7f8050068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:46.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8058083ff0 con 0x7f80700c1020 2023-10-30T12:45:46.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.945+0000 7f806e59c640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 0x7f8050068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:46.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.949+0000 7f806e59c640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 0x7f8050068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80600011f0 tx=0x7f8060009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:46.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:46.949+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f80580b5090 con 0x7f80700c1020 2023-10-30T12:45:47.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.109+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80700c99b0 con 0x7f80700c1020 2023-10-30T12:45:47.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.109+0000 7f8066ffd640 1 -- 172.21.15.8:0/3293258386 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8058021020 con 0x7f80700c1020 2023-10-30T12:45:47.114 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:45:47.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 msgr2=0x7f8050068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 0x7f8050068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80600011f0 tx=0x7f8060009040 comp rx=0 tx=0).stop 2023-10-30T12:45:47.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 msgr2=0x7f80700c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f805800e870 tx=0x7f805800ec70 comp rx=0 tx=0).stop 2023-10-30T12:45:47.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 shutdown_connections 2023-10-30T12:45:47.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f80500664a0 0x7f8050068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8070151a20 0x7f8070071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80700c1020 0x7f80700c9d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 --2- 172.21.15.8:0/3293258386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80700725c0 0x7f8070156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 >> 172.21.15.8:0/3293258386 conn(0x7f807006da40 msgr2=0x7f807014f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 shutdown_connections 2023-10-30T12:45:47.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.113+0000 7f8074b1d640 1 -- 172.21.15.8:0/3293258386 wait complete. 2023-10-30T12:45:47.135 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:45:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.277+0000 7fc4c03d3640 1 Processor -- start 2023-10-30T12:45:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.277+0000 7fc4c03d3640 1 -- start start 2023-10-30T12:45:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4c03d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4b80cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4c03d3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b8153f80 con 0x7fc4b80c4410 2023-10-30T12:45:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4c03d3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b8159df0 con 0x7fc4b80c4810 2023-10-30T12:45:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4c03d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b81597c0 con 0x7fc4b8153ba0 2023-10-30T12:45:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd96e640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fc4b80c4410 0x7fc4b815cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60504/0 (socket says 172.21.15.8:60504) 2023-10-30T12:45:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd96e640 1 -- 172.21.15.8:0/1767224050 learned_addr learned my addr 172.21.15.8:0/1767224050 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be16f640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4b80cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794101617 0 0) 0x7fc4b8153f80 con 0x7fc4b80c4410 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4003530 con 0x7fc4b80c4410 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043184481 0 0) 0x7fc4a4003530 con 0x7fc4b80c4410 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 msgr2=0x7fc4b80cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4b80cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 >> v1:172.21.15.46:6789/0 conn(0x7fc4b80c4810 legacy=0x7fc4b80cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b815d580 con 0x7fc4b80c4410 2023-10-30T12:45:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fc4a8001680 con 0x7fc4b80c4410 2023-10-30T12:45:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 >> v1:172.21.15.8:6789/0 conn(0x7fc4b80c4410 legacy=0x7fc4b815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4a4003ef0 0x7fc4a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b81597c0 con 0x7fc4a4004d10 2023-10-30T12:45:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a400baa0 con 0x7fc4a4003ef0 2023-10-30T12:45:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a400bc60 con 0x7fc4b8153ba0 2023-10-30T12:45:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be16f640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4bd96e640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4a4003ef0 0x7fc4a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 -- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 msgr2=0x7fc4a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 -- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4a4003ef0 msgr2=0x7fc4a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4a4003ef0 0x7fc4a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be970640 1 -- 172.21.15.8:0/1767224050 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a4003530 con 0x7fc4a4004d10 2023-10-30T12:45:47.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.281+0000 7fc4be16f640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b8153ba0 0x7fc4a40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:47.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4be970640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4a4005190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80c4c80 tx=0x7fc4b001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d12ea62359dad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a6660 con 0x7fc4a4004d10 2023-10-30T12:45:47.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00a67e0 con 0x7fc4a4004d10 2023-10-30T12:45:47.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4bd16d640 1 -- 172.21.15.8:0/1767224050 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a6b90 con 0x7fc4a4004d10 2023-10-30T12:45:47.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 -- 172.21.15.8:0/1767224050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 msgr2=0x7fc4a4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/1767224050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4a4005190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80c4c80 tx=0x7fc4b001c870 comp rx=0 tx=0).stop 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4be970640 1 -- 172.21.15.8:0/1767224050 reap_dead start 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 -- 172.21.15.8:0/1767224050 shutdown_connections 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 -- 172.21.15.8:0/1767224050 >> 172.21.15.8:0/1767224050 conn(0x7fc4b8071790 msgr2=0x7fc4b8071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 -- 172.21.15.8:0/1767224050 shutdown_connections 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.285+0000 7fc4c03d3640 1 -- 172.21.15.8:0/1767224050 wait complete. 2023-10-30T12:45:47.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 Processor -- start 2023-10-30T12:45:47.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- start start 2023-10-30T12:45:47.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b815b4a0 0x7fc4b80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 0x7fc4b81f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814fc10 con 0x7fc4a4004d10 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814f240 con 0x7fc4b80c8cb0 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b81f2c70 con 0x7fc4b815b4a0 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53416/0 (socket says 172.21.15.8:53416) 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 -- 172.21.15.8:0/338077454 learned_addr learned my addr 172.21.15.8:0/338077454 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:47.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4bd96e640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b815b4a0 0x7fc4b80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be970640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 0x7fc4b81f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 -- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b815b4a0 msgr2=0x7fc4b80b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b815b4a0 0x7fc4b80b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 -- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 msgr2=0x7fc4b81f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 0x7fc4b81f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 -- 172.21.15.8:0/338077454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b815a7f0 con 0x7fc4a4004d10 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be970640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 0x7fc4b81f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4be16f640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009fd0 tx=0x7fc4b4011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4004220 con 0x7fc4a4004d10 2023-10-30T12:45:47.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4004380 con 0x7fc4a4004d10 2023-10-30T12:45:47.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400f980 con 0x7fc4a4004d10 2023-10-30T12:45:47.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b81f26b0 con 0x7fc4a4004d10 2023-10-30T12:45:47.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b814ebf0 con 0x7fc4a4004d10 2023-10-30T12:45:47.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.289+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b804e170 con 0x7fc4a4004d10 2023-10-30T12:45:47.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.293+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4b400fae0 con 0x7fc4a4004d10 2023-10-30T12:45:47.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.293+0000 7fc4aeffd640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 0x7fc49c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.293+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b40844e0 con 0x7fc4a4004d10 2023-10-30T12:45:47.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.293+0000 7fc4bd96e640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 0x7fc49c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.293+0000 7fc4bd96e640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 0x7fc49c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4b81572a0 tx=0x7fc4a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.297+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4b40518c0 con 0x7fc4a4004d10 2023-10-30T12:45:47.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.453+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4b80cabb0 con 0x7fc4a4004d10 2023-10-30T12:45:47.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.453+0000 7fc4aeffd640 1 -- 172.21.15.8:0/338077454 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4b4054ee0 con 0x7fc4a4004d10 2023-10-30T12:45:47.457 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:45:47.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 msgr2=0x7fc49c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 0x7fc49c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4b81572a0 tx=0x7fc4a8009040 comp rx=0 tx=0).stop 2023-10-30T12:45:47.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 msgr2=0x7fc4b814f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009fd0 tx=0x7fc4b4011b30 comp rx=0 tx=0).stop 2023-10-30T12:45:47.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 shutdown_connections 2023-10-30T12:45:47.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc49c0664a0 0x7fc49c068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4b80c8cb0 0x7fc4b81f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc4b815b4a0 0x7fc4b80b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 --2- 172.21.15.8:0/338077454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4a4004d10 0x7fc4b814f6d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 >> 172.21.15.8:0/338077454 conn(0x7fc4b8071790 msgr2=0x7fc4b8164690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:47.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 shutdown_connections 2023-10-30T12:45:47.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.457+0000 7fc4c03d3640 1 -- 172.21.15.8:0/338077454 wait complete. 2023-10-30T12:45:47.478 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2023-10-30T12:45:47.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:47.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 Processor -- start 2023-10-30T12:45:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 -- start start 2023-10-30T12:45:47.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 0x7f97f40c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f415d0a0 con 0x7f97f4153300 2023-10-30T12:45:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f415d380 con 0x7f97f40718d0 2023-10-30T12:45:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f93ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40cca10 con 0x7f97f4071cb0 2023-10-30T12:45:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f37fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f97f4153300 0x7f97f40cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60518/0 (socket says 172.21.15.8:60518) 2023-10-30T12:45:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f37fe640 1 -- 172.21.15.8:0/1947766937 learned_addr learned my addr 172.21.15.8:0/1947766937 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f27fc640 1 --2- 172.21.15.8:0/1947766937 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 0x7f97f40c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904560524 0 0) 0x7f97f415d0a0 con 0x7f97f4153300 2023-10-30T12:45:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97dc003530 con 0x7f97f4153300 2023-10-30T12:45:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f27fc640 1 -- 172.21.15.8:0/1947766937 >> v1:172.21.15.46:6789/0 conn(0x7f97f40718d0 legacy=0x7f97f415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153502123 0 0) 0x7f97f415d380 con 0x7f97f40718d0 2023-10-30T12:45:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f27fc640 1 -- 172.21.15.8:0/1947766937 >> v1:172.21.15.8:6789/0 conn(0x7f97f4153300 legacy=0x7f97f40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.621+0000 7f97f27fc640 1 -- 172.21.15.8:0/1947766937 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f415d660 con 0x7f97f4071cb0 2023-10-30T12:45:47.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f27fc640 1 --2- 172.21.15.8:0/1947766937 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 0x7f97f40c9a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97e80097f0 tx=0x7f97e801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c5b0039e39430 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80a89b0 con 0x7f97f4071cb0 2023-10-30T12:45:47.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e80a8b10 con 0x7f97f4071cb0 2023-10-30T12:45:47.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f1ffb640 1 -- 172.21.15.8:0/1947766937 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80a8e80 con 0x7f97f4071cb0 2023-10-30T12:45:47.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 -- 172.21.15.8:0/1947766937 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 msgr2=0x7f97f40c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 --2- 172.21.15.8:0/1947766937 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 0x7f97f40c9a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97e80097f0 tx=0x7f97e801ecb0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 -- 172.21.15.8:0/1947766937 shutdown_connections 2023-10-30T12:45:47.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 --2- 172.21.15.8:0/1947766937 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4071cb0 0x7f97f40c9a10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 -- 172.21.15.8:0/1947766937 >> 172.21.15.8:0/1947766937 conn(0x7f97f406ab00 msgr2=0x7f97f4065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:47.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 -- 172.21.15.8:0/1947766937 shutdown_connections 2023-10-30T12:45:47.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.625+0000 7f97f93ab640 1 -- 172.21.15.8:0/1947766937 wait complete. 2023-10-30T12:45:47.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 Processor -- start 2023-10-30T12:45:47.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- start start 2023-10-30T12:45:47.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4071cb0 0x7f97f41f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4153300 0x7f97f41f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f414d810 con 0x7f97f40718d0 2023-10-30T12:45:47.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f414d440 con 0x7f97f4071cb0 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40b3970 con 0x7f97f4153300 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53418/0 (socket says 172.21.15.8:53418) 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 -- 172.21.15.8:0/3047025868 learned_addr learned my addr 172.21.15.8:0/3047025868 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f37fe640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4153300 0x7f97f41f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f27fc640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4071cb0 0x7f97f41f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 -- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4153300 msgr2=0x7f97f41f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4153300 0x7f97f41f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 -- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4071cb0 msgr2=0x7f97f41f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4071cb0 0x7f97f41f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 -- 172.21.15.8:0/3047025868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009420 con 0x7f97f40718d0 2023-10-30T12:45:47.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f2ffd640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c4b0 tx=0x7f97e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4005480 con 0x7f97f40718d0 2023-10-30T12:45:47.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4003600 con 0x7f97f40718d0 2023-10-30T12:45:47.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f40b1980 con 0x7f97f40718d0 2023-10-30T12:45:47.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40039b0 con 0x7f97f40718d0 2023-10-30T12:45:47.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.629+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f414da20 con 0x7f97f40718d0 2023-10-30T12:45:47.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f41f1ef0 con 0x7f97f40718d0 2023-10-30T12:45:47.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97e4003b10 con 0x7f97f40718d0 2023-10-30T12:45:47.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97db7fe640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 0x7f97e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e4083a00 con 0x7f97f40718d0 2023-10-30T12:45:47.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97f27fc640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 0x7f97e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.633+0000 7f97f27fc640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 0x7f97e0068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b2000 tx=0x7f97e80aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.637+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f97e4050e00 con 0x7f97f40718d0 2023-10-30T12:45:47.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.825+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f40b3ae0 con 0x7f97f40718d0 2023-10-30T12:45:47.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.825+0000 7f97db7fe640 1 -- 172.21.15.8:0/3047025868 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f97e4054420 con 0x7f97f40718d0 2023-10-30T12:45:47.829 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:47.829 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:47.829 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 msgr2=0x7f97e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 0x7f97e0068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b2000 tx=0x7f97e80aa070 comp rx=0 tx=0).stop 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 msgr2=0x7f97f4149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c4b0 tx=0x7f97e400c970 comp rx=0 tx=0).stop 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 shutdown_connections 2023-10-30T12:45:47.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97e00664a0 0x7f97e0068960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4071cb0 0x7f97f41f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f97f4153300 0x7f97f41f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 --2- 172.21.15.8:0/3047025868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97f40718d0 0x7f97f4149c10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 >> 172.21.15.8:0/3047025868 conn(0x7f97f406ab00 msgr2=0x7f97f40cbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:47.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 shutdown_connections 2023-10-30T12:45:47.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.829+0000 7f97f93ab640 1 -- 172.21.15.8:0/3047025868 wait complete. 2023-10-30T12:45:47.850 INFO:tasks.mds_thrash:Ready to start thrashing 2023-10-30T12:45:47.851 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2023-10-30T12:45:47.851 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.989+0000 7f8cad691640 1 Processor -- start 2023-10-30T12:45:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.989+0000 7f8cad691640 1 -- start start 2023-10-30T12:45:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ca8153c80 0x7f8ca815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8ca80c4a10 0x7f8ca80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca815a3b0 0x7f8ca80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca815dec0 con 0x7f8ca80c4a10 2023-10-30T12:45:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca81599a0 con 0x7f8ca8153c80 2023-10-30T12:45:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8cad691640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8159f20 con 0x7f8ca815a3b0 2023-10-30T12:45:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8ca80c4a10 0x7f8ca80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8ca80c4a10 0x7f8ca80c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53428/0 (socket says 172.21.15.8:53428) 2023-10-30T12:45:47.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 -- 172.21.15.8:0/235200862 learned_addr learned my addr 172.21.15.8:0/235200862 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:47.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8ca77fe640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca815a3b0 0x7f8ca80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8ca6ffd640 1 --2- 172.21.15.8:0/235200862 >> v2:172.21.15.46:3300/0 conn(0x7f8ca8153c80 0x7f8ca815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:47.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 -- 172.21.15.8:0/235200862 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca815a3b0 msgr2=0x7f8ca80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:47.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca815a3b0 0x7f8ca80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 -- 172.21.15.8:0/235200862 >> v2:172.21.15.46:3300/0 conn(0x7f8ca8153c80 msgr2=0x7f8ca815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:47.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/235200862 >> v2:172.21.15.46:3300/0 conn(0x7f8ca8153c80 0x7f8ca815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:47.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.993+0000 7f8c9ffff640 1 -- 172.21.15.8:0/235200862 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80c47e0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca80c42a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d00f0 tx=0x7f8c9001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be54f56bfe2805a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8ca67fc640 1 -- 172.21.15.8:0/235200862 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900a89b0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8ca67fc640 1 -- 172.21.15.8:0/235200862 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c900a8b10 con 0x7f8ca80c4a10 2023-10-30T12:45:48.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8ca67fc640 1 -- 172.21.15.8:0/235200862 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900a8ea0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- 172.21.15.8:0/235200862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 msgr2=0x7f8ca80c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca80c42a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d00f0 tx=0x7f8c9001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- 172.21.15.8:0/235200862 shutdown_connections 2023-10-30T12:45:48.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca815a3b0 0x7f8ca80ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 --2- 172.21.15.8:0/235200862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca80c42a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 --2- 172.21.15.8:0/235200862 >> v2:172.21.15.46:3300/0 conn(0x7f8ca8153c80 0x7f8ca815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- 172.21.15.8:0/235200862 >> 172.21.15.8:0/235200862 conn(0x7f8ca80715c0 msgr2=0x7f8ca80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- 172.21.15.8:0/235200862 shutdown_connections 2023-10-30T12:45:48.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- 172.21.15.8:0/235200862 wait complete. 2023-10-30T12:45:48.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 Processor -- start 2023-10-30T12:45:48.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:47.997+0000 7f8cad691640 1 -- start start 2023-10-30T12:45:48.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca8153c80 0x7f8ca80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca815a3b0 0x7f8ca8147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81f2c20 con 0x7f8ca80c4a10 2023-10-30T12:45:48.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8148620 con 0x7f8ca815a3b0 2023-10-30T12:45:48.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81ee870 con 0x7f8ca8153c80 2023-10-30T12:45:48.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53440/0 (socket says 172.21.15.8:53440) 2023-10-30T12:45:48.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 -- 172.21.15.8:0/2484699743 learned_addr learned my addr 172.21.15.8:0/2484699743 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:48.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca8153c80 0x7f8ca80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca77fe640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca815a3b0 0x7f8ca8147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 -- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca8153c80 msgr2=0x7f8ca80b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca8153c80 0x7f8ca80b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 -- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca815a3b0 msgr2=0x7f8ca8147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca815a3b0 0x7f8ca8147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 -- 172.21.15.8:0/2484699743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900094d0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8ca6ffd640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800d800 tx=0x7f8c9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980096a0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98009800 con 0x7f8ca80c4a10 2023-10-30T12:45:48.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98009b70 con 0x7f8ca80c4a10 2023-10-30T12:45:48.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca81eeb50 con 0x7f8ca80c4a10 2023-10-30T12:45:48.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca81ef110 con 0x7f8ca80c4a10 2023-10-30T12:45:48.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca804e170 con 0x7f8ca80c4a10 2023-10-30T12:45:48.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.001+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8c98009cd0 con 0x7f8ca80c4a10 2023-10-30T12:45:48.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.005+0000 7f8c9f7fe640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 0x7f8c88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.005+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c98084730 con 0x7f8ca80c4a10 2023-10-30T12:45:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.005+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 0x7f8c88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.005+0000 7f8c9ffff640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 0x7f8c88068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c900b2000 tx=0x7f8c90008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.009+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8c98051b10 con 0x7f8ca80c4a10 2023-10-30T12:45:48.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.161+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ca815d900 con 0x7f8c880664a0 2023-10-30T12:45:48.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.169+0000 7f8c9f7fe640 1 -- 172.21.15.8:0/2484699743 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264634 (secure 0 0 0) 0x7f8c900b0440 con 0x7f8c880664a0 2023-10-30T12:45:48.174 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:48.176 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 msgr2=0x7f8c88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 0x7f8c88068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c900b2000 tx=0x7f8c90008040 comp rx=0 tx=0).stop 2023-10-30T12:45:48.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 msgr2=0x7f8ca81f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800d800 tx=0x7f8c9800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 shutdown_connections 2023-10-30T12:45:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8c880664a0 0x7f8c88068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca815a3b0 0x7f8ca8147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8ca8153c80 0x7f8ca80b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 --2- 172.21.15.8:0/2484699743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ca80c4a10 0x7f8ca81f26e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 >> 172.21.15.8:0/2484699743 conn(0x7f8ca80715c0 msgr2=0x7f8ca80c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 shutdown_connections 2023-10-30T12:45:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.173+0000 7f8cad691640 1 -- 172.21.15.8:0/2484699743 wait complete. 2023-10-30T12:45:48.195 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-30T12:45:47.618606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252324,"kb_used_data":2916,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722396,"statfs":{"total":773094113280,"available":772835733504,"internally_reserved":0,"allocated":2985984,"data_stored":1759192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071955+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:20.071955+0000","last_peered":"2023-10-30T12:45:20.071955+0000","last_clean":"2023-10-30T12:45:20.071955+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:20.071955+0000","last_undegraded":"2023-10-30T12:45:20.071955+0000","last_fullsized":"2023-10-30T12:45:20.071955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:33:48.910689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067349+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:20.067349+0000","last_peered":"2023-10-30T12:45:20.067349+0000","last_clean":"2023-10-30T12:45:20.067349+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:20.067349+0000","last_undegraded":"2023-10-30T12:45:20.067349+0000","last_fullsized":"2023-10-30T12:45:20.067349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:34.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470964+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:20.470964+0000","last_peered":"2023-10-30T12:45:20.470964+0000","last_clean":"2023-10-30T12:45:20.470964+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:20.470964+0000","last_undegraded":"2023-10-30T12:45:20.470964+0000","last_fullsized":"2023-10-30T12:45:20.470964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:45:42.325441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377353+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:20.377353+0000","last_peered":"2023-10-30T12:45:20.377353+0000","last_clean":"2023-10-30T12:45:20.377353+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:20.377353+0000","last_undegraded":"2023-10-30T12:45:20.377353+0000","last_fullsized":"2023-10-30T12:45:20.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:59.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069882+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:20.069882+0000","last_peered":"2023-10-30T12:45:20.069882+0000","last_clean":"2023-10-30T12:45:20.069882+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:20.069882+0000","last_undegraded":"2023-10-30T12:45:20.069882+0000","last_fullsized":"2023-10-30T12:45:20.069882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:09.263782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376997+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:20.376997+0000","last_peered":"2023-10-30T12:45:20.376997+0000","last_clean":"2023-10-30T12:45:20.376997+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:20.376997+0000","last_undegraded":"2023-10-30T12:45:20.376997+0000","last_fullsized":"2023-10-30T12:45:20.376997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:48:04.677113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069348+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:20.069348+0000","last_peered":"2023-10-30T12:45:20.069348+0000","last_clean":"2023-10-30T12:45:20.069348+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:20.069348+0000","last_undegraded":"2023-10-30T12:45:20.069348+0000","last_fullsized":"2023-10-30T12:45:20.069348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:43:52.826848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470720+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:20.470720+0000","last_peered":"2023-10-30T12:45:20.470720+0000","last_clean":"2023-10-30T12:45:20.470720+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:20.470720+0000","last_undegraded":"2023-10-30T12:45:20.470720+0000","last_fullsized":"2023-10-30T12:45:20.470720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:27:17.979749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072297+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:20.072297+0000","last_peered":"2023-10-30T12:45:20.072297+0000","last_clean":"2023-10-30T12:45:20.072297+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:20.072297+0000","last_undegraded":"2023-10-30T12:45:20.072297+0000","last_fullsized":"2023-10-30T12:45:20.072297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:28.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471673+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:20.471673+0000","last_peered":"2023-10-30T12:45:20.471673+0000","last_clean":"2023-10-30T12:45:20.471673+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:20.471673+0000","last_undegraded":"2023-10-30T12:45:20.471673+0000","last_fullsized":"2023-10-30T12:45:20.471673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:06:39.736671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471336+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:20.471336+0000","last_peered":"2023-10-30T12:45:20.471336+0000","last_clean":"2023-10-30T12:45:20.471336+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:20.471336+0000","last_undegraded":"2023-10-30T12:45:20.471336+0000","last_fullsized":"2023-10-30T12:45:20.471336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:55.158597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071808+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:20.071808+0000","last_peered":"2023-10-30T12:45:20.071808+0000","last_clean":"2023-10-30T12:45:20.071808+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:20.071808+0000","last_undegraded":"2023-10-30T12:45:20.071808+0000","last_fullsized":"2023-10-30T12:45:20.071808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:18:07.085147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470864+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:20.470864+0000","last_peered":"2023-10-30T12:45:20.470864+0000","last_clean":"2023-10-30T12:45:20.470864+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:20.470864+0000","last_undegraded":"2023-10-30T12:45:20.470864+0000","last_fullsized":"2023-10-30T12:45:20.470864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:45:04.008631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470638+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:20.470638+0000","last_peered":"2023-10-30T12:45:20.470638+0000","last_clean":"2023-10-30T12:45:20.470638+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:20.470638+0000","last_undegraded":"2023-10-30T12:45:20.470638+0000","last_fullsized":"2023-10-30T12:45:20.470638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:02:04.825098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:23.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376825+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:20.376825+0000","last_peered":"2023-10-30T12:45:20.376825+0000","last_clean":"2023-10-30T12:45:20.376825+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:20.376825+0000","last_undegraded":"2023-10-30T12:45:20.376825+0000","last_fullsized":"2023-10-30T12:45:20.376825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:48:16.344709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377969+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:20.377969+0000","last_peered":"2023-10-30T12:45:20.377969+0000","last_clean":"2023-10-30T12:45:20.377969+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:20.377969+0000","last_undegraded":"2023-10-30T12:45:20.377969+0000","last_fullsized":"2023-10-30T12:45:20.377969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:08.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471127+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:20.471127+0000","last_peered":"2023-10-30T12:45:20.471127+0000","last_clean":"2023-10-30T12:45:20.471127+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:20.471127+0000","last_undegraded":"2023-10-30T12:45:20.471127+0000","last_fullsized":"2023-10-30T12:45:20.471127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:05.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070768+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:20.070768+0000","last_peered":"2023-10-30T12:45:20.070768+0000","last_clean":"2023-10-30T12:45:20.070768+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:20.070768+0000","last_undegraded":"2023-10-30T12:45:20.070768+0000","last_fullsized":"2023-10-30T12:45:20.070768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:17.483158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471411+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:20.471411+0000","last_peered":"2023-10-30T12:45:20.471411+0000","last_clean":"2023-10-30T12:45:20.471411+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:20.471411+0000","last_undegraded":"2023-10-30T12:45:20.471411+0000","last_fullsized":"2023-10-30T12:45:20.471411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:16:21.392983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470962+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:20.470962+0000","last_peered":"2023-10-30T12:45:20.470962+0000","last_clean":"2023-10-30T12:45:20.470962+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:20.470962+0000","last_undegraded":"2023-10-30T12:45:20.470962+0000","last_fullsized":"2023-10-30T12:45:20.470962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:02.564276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377142+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:20.377142+0000","last_peered":"2023-10-30T12:45:20.377142+0000","last_clean":"2023-10-30T12:45:20.377142+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:20.377142+0000","last_undegraded":"2023-10-30T12:45:20.377142+0000","last_fullsized":"2023-10-30T12:45:20.377142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:33:03.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470206+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:20.470206+0000","last_peered":"2023-10-30T12:45:20.470206+0000","last_clean":"2023-10-30T12:45:20.470206+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:20.470206+0000","last_undegraded":"2023-10-30T12:45:20.470206+0000","last_fullsized":"2023-10-30T12:45:20.470206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:31:49.660950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376996+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:20.376996+0000","last_peered":"2023-10-30T12:45:20.376996+0000","last_clean":"2023-10-30T12:45:20.376996+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:20.376996+0000","last_undegraded":"2023-10-30T12:45:20.376996+0000","last_fullsized":"2023-10-30T12:45:20.376996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:51.286544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471712+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:20.471712+0000","last_peered":"2023-10-30T12:45:20.471712+0000","last_clean":"2023-10-30T12:45:20.471712+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:20.471712+0000","last_undegraded":"2023-10-30T12:45:20.471712+0000","last_fullsized":"2023-10-30T12:45:20.471712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:57.636148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471671+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:20.471671+0000","last_peered":"2023-10-30T12:45:20.471671+0000","last_clean":"2023-10-30T12:45:20.471671+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:20.471671+0000","last_undegraded":"2023-10-30T12:45:20.471671+0000","last_fullsized":"2023-10-30T12:45:20.471671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:30.410457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067039+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:20.067039+0000","last_peered":"2023-10-30T12:45:20.067039+0000","last_clean":"2023-10-30T12:45:20.067039+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:20.067039+0000","last_undegraded":"2023-10-30T12:45:20.067039+0000","last_fullsized":"2023-10-30T12:45:20.067039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:46.788124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471494+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:20.471494+0000","last_peered":"2023-10-30T12:45:20.471494+0000","last_clean":"2023-10-30T12:45:20.471494+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:20.471494+0000","last_undegraded":"2023-10-30T12:45:20.471494+0000","last_fullsized":"2023-10-30T12:45:20.471494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:44.555138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471084+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:20.471084+0000","last_peered":"2023-10-30T12:45:20.471084+0000","last_clean":"2023-10-30T12:45:20.471084+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:20.471084+0000","last_undegraded":"2023-10-30T12:45:20.471084+0000","last_fullsized":"2023-10-30T12:45:20.471084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:29.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377430+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:20.377430+0000","last_peered":"2023-10-30T12:45:20.377430+0000","last_clean":"2023-10-30T12:45:20.377430+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:20.377430+0000","last_undegraded":"2023-10-30T12:45:20.377430+0000","last_fullsized":"2023-10-30T12:45:20.377430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:50:53.563609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471353+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:20.471353+0000","last_peered":"2023-10-30T12:45:20.471353+0000","last_clean":"2023-10-30T12:45:20.471353+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:20.471353+0000","last_undegraded":"2023-10-30T12:45:20.471353+0000","last_fullsized":"2023-10-30T12:45:20.471353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:12:59.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470912+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:20.470912+0000","last_peered":"2023-10-30T12:45:20.470912+0000","last_clean":"2023-10-30T12:45:20.470912+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:20.470912+0000","last_undegraded":"2023-10-30T12:45:20.470912+0000","last_fullsized":"2023-10-30T12:45:20.470912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:10.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071561+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:20.071561+0000","last_peered":"2023-10-30T12:45:20.071561+0000","last_clean":"2023-10-30T12:45:20.071561+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:20.071561+0000","last_undegraded":"2023-10-30T12:45:20.071561+0000","last_fullsized":"2023-10-30T12:45:20.071561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:17:56.183441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.167894+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:29.167894+0000","last_peered":"2023-10-30T12:45:29.167894+0000","last_clean":"2023-10-30T12:45:29.167894+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:29.167894+0000","last_undegraded":"2023-10-30T12:45:29.167894+0000","last_fullsized":"2023-10-30T12:45:29.167894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471141+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:20.471141+0000","last_peered":"2023-10-30T12:45:20.471141+0000","last_clean":"2023-10-30T12:45:20.471141+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:20.471141+0000","last_undegraded":"2023-10-30T12:45:20.471141+0000","last_fullsized":"2023-10-30T12:45:20.471141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:23:00.831740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203892+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:35.203892+0000","last_peered":"2023-10-30T12:45:35.203892+0000","last_clean":"2023-10-30T12:45:35.203892+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:35.203892+0000","last_undegraded":"2023-10-30T12:45:35.203892+0000","last_fullsized":"2023-10-30T12:45:35.203892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203844+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:35.203844+0000","last_peered":"2023-10-30T12:45:35.203844+0000","last_clean":"2023-10-30T12:45:35.203844+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:35.203844+0000","last_undegraded":"2023-10-30T12:45:35.203844+0000","last_fullsized":"2023-10-30T12:45:35.203844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470208+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:20.470208+0000","last_peered":"2023-10-30T12:45:20.470208+0000","last_clean":"2023-10-30T12:45:20.470208+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:20.470208+0000","last_undegraded":"2023-10-30T12:45:20.470208+0000","last_fullsized":"2023-10-30T12:45:20.470208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:26.174484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377740+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:20.377740+0000","last_peered":"2023-10-30T12:45:20.377740+0000","last_clean":"2023-10-30T12:45:20.377740+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:20.377740+0000","last_undegraded":"2023-10-30T12:45:20.377740+0000","last_fullsized":"2023-10-30T12:45:20.377740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:12:12.444137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157838+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:29.157838+0000","last_peered":"2023-10-30T12:45:29.157838+0000","last_clean":"2023-10-30T12:45:29.157838+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:29.157838+0000","last_undegraded":"2023-10-30T12:45:29.157838+0000","last_fullsized":"2023-10-30T12:45:29.157838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377703+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:20.377703+0000","last_peered":"2023-10-30T12:45:20.377703+0000","last_clean":"2023-10-30T12:45:20.377703+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:20.377703+0000","last_undegraded":"2023-10-30T12:45:20.377703+0000","last_fullsized":"2023-10-30T12:45:20.377703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:40:41.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.158692+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:29.158692+0000","last_peered":"2023-10-30T12:45:29.158692+0000","last_clean":"2023-10-30T12:45:29.158692+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:29.158692+0000","last_undegraded":"2023-10-30T12:45:29.158692+0000","last_fullsized":"2023-10-30T12:45:29.158692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471307+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:20.471307+0000","last_peered":"2023-10-30T12:45:20.471307+0000","last_clean":"2023-10-30T12:45:20.471307+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:20.471307+0000","last_undegraded":"2023-10-30T12:45:20.471307+0000","last_fullsized":"2023-10-30T12:45:20.471307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:07:09.393386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.086171+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:24.086171+0000","last_peered":"2023-10-30T12:45:24.086171+0000","last_clean":"2023-10-30T12:45:24.086171+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:24.086171+0000","last_undegraded":"2023-10-30T12:45:24.086171+0000","last_fullsized":"2023-10-30T12:45:24.086171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072148+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:20.072148+0000","last_peered":"2023-10-30T12:45:20.072148+0000","last_clean":"2023-10-30T12:45:20.072148+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:20.072148+0000","last_undegraded":"2023-10-30T12:45:20.072148+0000","last_fullsized":"2023-10-30T12:45:20.072148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:17.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'11","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:43.496620+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:43.496620+0000","last_peered":"2023-10-30T12:45:43.496620+0000","last_clean":"2023-10-30T12:45:43.496620+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:43.496620+0000","last_undegraded":"2023-10-30T12:45:43.496620+0000","last_fullsized":"2023-10-30T12:45:43.496620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069876+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:20.069876+0000","last_peered":"2023-10-30T12:45:20.069876+0000","last_clean":"2023-10-30T12:45:20.069876+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:20.069876+0000","last_undegraded":"2023-10-30T12:45:20.069876+0000","last_fullsized":"2023-10-30T12:45:20.069876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:07:42.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201182+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:35.201182+0000","last_peered":"2023-10-30T12:45:35.201182+0000","last_clean":"2023-10-30T12:45:35.201182+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:35.201182+0000","last_undegraded":"2023-10-30T12:45:35.201182+0000","last_fullsized":"2023-10-30T12:45:35.201182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470757+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:20.470757+0000","last_peered":"2023-10-30T12:45:20.470757+0000","last_clean":"2023-10-30T12:45:20.470757+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:20.470757+0000","last_undegraded":"2023-10-30T12:45:20.470757+0000","last_fullsized":"2023-10-30T12:45:20.470757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:17.831426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.164676+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:29.164676+0000","last_peered":"2023-10-30T12:45:29.164676+0000","last_clean":"2023-10-30T12:45:29.164676+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:29.164676+0000","last_undegraded":"2023-10-30T12:45:29.164676+0000","last_fullsized":"2023-10-30T12:45:29.164676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070431+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:20.070431+0000","last_peered":"2023-10-30T12:45:20.070431+0000","last_clean":"2023-10-30T12:45:20.070431+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:20.070431+0000","last_undegraded":"2023-10-30T12:45:20.070431+0000","last_fullsized":"2023-10-30T12:45:20.070431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:14:31.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.210377+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:35.210377+0000","last_peered":"2023-10-30T12:45:35.210377+0000","last_clean":"2023-10-30T12:45:35.210377+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:35.210377+0000","last_undegraded":"2023-10-30T12:45:35.210377+0000","last_fullsized":"2023-10-30T12:45:35.210377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470957+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:20.470957+0000","last_peered":"2023-10-30T12:45:20.470957+0000","last_clean":"2023-10-30T12:45:20.470957+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:20.470957+0000","last_undegraded":"2023-10-30T12:45:20.470957+0000","last_fullsized":"2023-10-30T12:45:20.470957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:13.282854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'15","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201839+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:35.201839+0000","last_peered":"2023-10-30T12:45:35.201839+0000","last_clean":"2023-10-30T12:45:35.201839+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:35.201839+0000","last_undegraded":"2023-10-30T12:45:35.201839+0000","last_fullsized":"2023-10-30T12:45:35.201839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069448+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:20.069448+0000","last_peered":"2023-10-30T12:45:20.069448+0000","last_clean":"2023-10-30T12:45:20.069448+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:20.069448+0000","last_undegraded":"2023-10-30T12:45:20.069448+0000","last_fullsized":"2023-10-30T12:45:20.069448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:14.563685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.079122+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:24.079122+0000","last_peered":"2023-10-30T12:45:24.079122+0000","last_clean":"2023-10-30T12:45:24.079122+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:24.079122+0000","last_undegraded":"2023-10-30T12:45:24.079122+0000","last_fullsized":"2023-10-30T12:45:24.079122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071276+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:20.071276+0000","last_peered":"2023-10-30T12:45:20.071276+0000","last_clean":"2023-10-30T12:45:20.071276+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:20.071276+0000","last_undegraded":"2023-10-30T12:45:20.071276+0000","last_fullsized":"2023-10-30T12:45:20.071276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:43.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.077908+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:24.077908+0000","last_peered":"2023-10-30T12:45:24.077908+0000","last_clean":"2023-10-30T12:45:24.077908+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:24.077908+0000","last_undegraded":"2023-10-30T12:45:24.077908+0000","last_fullsized":"2023-10-30T12:45:24.077908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072543+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:20.072543+0000","last_peered":"2023-10-30T12:45:20.072543+0000","last_clean":"2023-10-30T12:45:20.072543+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:20.072543+0000","last_undegraded":"2023-10-30T12:45:20.072543+0000","last_fullsized":"2023-10-30T12:45:20.072543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:54.641020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067271+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:20.067271+0000","last_peered":"2023-10-30T12:45:20.067271+0000","last_clean":"2023-10-30T12:45:20.067271+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:20.067271+0000","last_undegraded":"2023-10-30T12:45:20.067271+0000","last_fullsized":"2023-10-30T12:45:20.067271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378262+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:20.378262+0000","last_peered":"2023-10-30T12:45:20.378262+0000","last_clean":"2023-10-30T12:45:20.378262+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:20.378262+0000","last_undegraded":"2023-10-30T12:45:20.378262+0000","last_fullsized":"2023-10-30T12:45:20.378262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:08.395587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.205713+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:35.205713+0000","last_peered":"2023-10-30T12:45:35.205713+0000","last_clean":"2023-10-30T12:45:35.205713+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:35.205713+0000","last_undegraded":"2023-10-30T12:45:35.205713+0000","last_fullsized":"2023-10-30T12:45:35.205713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069897+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:20.069897+0000","last_peered":"2023-10-30T12:45:20.069897+0000","last_clean":"2023-10-30T12:45:20.069897+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:20.069897+0000","last_undegraded":"2023-10-30T12:45:20.069897+0000","last_fullsized":"2023-10-30T12:45:20.069897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:58:00.569432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071157+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:19.071157+0000","last_peered":"2023-10-30T12:45:19.071157+0000","last_clean":"2023-10-30T12:45:19.071157+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:19.071157+0000","last_undegraded":"2023-10-30T12:45:19.071157+0000","last_fullsized":"2023-10-30T12:45:19.071157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156996+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:29.156996+0000","last_peered":"2023-10-30T12:45:29.156996+0000","last_clean":"2023-10-30T12:45:29.156996+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:29.156996+0000","last_undegraded":"2023-10-30T12:45:29.156996+0000","last_fullsized":"2023-10-30T12:45:29.156996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470630+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:20.470630+0000","last_peered":"2023-10-30T12:45:20.470630+0000","last_clean":"2023-10-30T12:45:20.470630+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:20.470630+0000","last_undegraded":"2023-10-30T12:45:20.470630+0000","last_fullsized":"2023-10-30T12:45:20.470630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:25:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201698+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:35.201698+0000","last_peered":"2023-10-30T12:45:35.201698+0000","last_clean":"2023-10-30T12:45:35.201698+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:35.201698+0000","last_undegraded":"2023-10-30T12:45:35.201698+0000","last_fullsized":"2023-10-30T12:45:35.201698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470604+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:20.470604+0000","last_peered":"2023-10-30T12:45:20.470604+0000","last_clean":"2023-10-30T12:45:20.470604+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:20.470604+0000","last_undegraded":"2023-10-30T12:45:20.470604+0000","last_fullsized":"2023-10-30T12:45:20.470604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:55.822788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.202222+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:35.202222+0000","last_peered":"2023-10-30T12:45:35.202222+0000","last_clean":"2023-10-30T12:45:35.202222+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:35.202222+0000","last_undegraded":"2023-10-30T12:45:35.202222+0000","last_fullsized":"2023-10-30T12:45:35.202222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066971+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:20.066971+0000","last_peered":"2023-10-30T12:45:20.066971+0000","last_clean":"2023-10-30T12:45:20.066971+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:20.066971+0000","last_undegraded":"2023-10-30T12:45:20.066971+0000","last_fullsized":"2023-10-30T12:45:20.066971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:17:16.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377841+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:20.377841+0000","last_peered":"2023-10-30T12:45:20.377841+0000","last_clean":"2023-10-30T12:45:20.377841+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:20.377841+0000","last_undegraded":"2023-10-30T12:45:20.377841+0000","last_fullsized":"2023-10-30T12:45:20.377841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072149+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:20.072149+0000","last_peered":"2023-10-30T12:45:20.072149+0000","last_clean":"2023-10-30T12:45:20.072149+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:20.072149+0000","last_undegraded":"2023-10-30T12:45:20.072149+0000","last_fullsized":"2023-10-30T12:45:20.072149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630601+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.630601+0000","last_peered":"2023-10-30T12:45:19.630601+0000","last_clean":"2023-10-30T12:45:19.630601+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.630601+0000","last_undegraded":"2023-10-30T12:45:19.630601+0000","last_fullsized":"2023-10-30T12:45:19.630601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378093+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:20.378093+0000","last_peered":"2023-10-30T12:45:20.378093+0000","last_clean":"2023-10-30T12:45:20.378093+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:20.378093+0000","last_undegraded":"2023-10-30T12:45:20.378093+0000","last_fullsized":"2023-10-30T12:45:20.378093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:05.357577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071109+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:20.071109+0000","last_peered":"2023-10-30T12:45:20.071109+0000","last_clean":"2023-10-30T12:45:20.071109+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:20.071109+0000","last_undegraded":"2023-10-30T12:45:20.071109+0000","last_fullsized":"2023-10-30T12:45:20.071109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066974+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:20.066974+0000","last_peered":"2023-10-30T12:45:20.066974+0000","last_clean":"2023-10-30T12:45:20.066974+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:20.066974+0000","last_undegraded":"2023-10-30T12:45:20.066974+0000","last_fullsized":"2023-10-30T12:45:20.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:46:43.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201094+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:35.201094+0000","last_peered":"2023-10-30T12:45:35.201094+0000","last_clean":"2023-10-30T12:45:35.201094+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:35.201094+0000","last_undegraded":"2023-10-30T12:45:35.201094+0000","last_fullsized":"2023-10-30T12:45:35.201094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471700+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:20.471700+0000","last_peered":"2023-10-30T12:45:20.471700+0000","last_clean":"2023-10-30T12:45:20.471700+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:20.471700+0000","last_undegraded":"2023-10-30T12:45:20.471700+0000","last_fullsized":"2023-10-30T12:45:20.471700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:25.320280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157353+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:29.157353+0000","last_peered":"2023-10-30T12:45:29.157353+0000","last_clean":"2023-10-30T12:45:29.157353+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:29.157353+0000","last_undegraded":"2023-10-30T12:45:29.157353+0000","last_fullsized":"2023-10-30T12:45:29.157353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471488+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:20.471488+0000","last_peered":"2023-10-30T12:45:20.471488+0000","last_clean":"2023-10-30T12:45:20.471488+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:20.471488+0000","last_undegraded":"2023-10-30T12:45:20.471488+0000","last_fullsized":"2023-10-30T12:45:20.471488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:14.407746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156619+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:29.156619+0000","last_peered":"2023-10-30T12:45:29.156619+0000","last_clean":"2023-10-30T12:45:29.156619+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:29.156619+0000","last_undegraded":"2023-10-30T12:45:29.156619+0000","last_fullsized":"2023-10-30T12:45:29.156619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071773+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:20.071773+0000","last_peered":"2023-10-30T12:45:20.071773+0000","last_clean":"2023-10-30T12:45:20.071773+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:20.071773+0000","last_undegraded":"2023-10-30T12:45:20.071773+0000","last_fullsized":"2023-10-30T12:45:20.071773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:09.037055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.165872+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:29.165872+0000","last_peered":"2023-10-30T12:45:29.165872+0000","last_clean":"2023-10-30T12:45:29.165872+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:29.165872+0000","last_undegraded":"2023-10-30T12:45:29.165872+0000","last_fullsized":"2023-10-30T12:45:29.165872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470363+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:20.470363+0000","last_peered":"2023-10-30T12:45:20.470363+0000","last_clean":"2023-10-30T12:45:20.470363+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:20.470363+0000","last_undegraded":"2023-10-30T12:45:20.470363+0000","last_fullsized":"2023-10-30T12:45:20.470363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:45:36.069824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067111+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:20.067111+0000","last_peered":"2023-10-30T12:45:20.067111+0000","last_clean":"2023-10-30T12:45:20.067111+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:20.067111+0000","last_undegraded":"2023-10-30T12:45:20.067111+0000","last_fullsized":"2023-10-30T12:45:20.067111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:21.343385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.151892+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:29.151892+0000","last_peered":"2023-10-30T12:45:29.151892+0000","last_clean":"2023-10-30T12:45:29.151892+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:29.151892+0000","last_undegraded":"2023-10-30T12:45:29.151892+0000","last_fullsized":"2023-10-30T12:45:29.151892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470979+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:20.470979+0000","last_peered":"2023-10-30T12:45:20.470979+0000","last_clean":"2023-10-30T12:45:20.470979+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:20.470979+0000","last_undegraded":"2023-10-30T12:45:20.470979+0000","last_fullsized":"2023-10-30T12:45:20.470979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:44.769458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377742+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:20.377742+0000","last_peered":"2023-10-30T12:45:20.377742+0000","last_clean":"2023-10-30T12:45:20.377742+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:20.377742+0000","last_undegraded":"2023-10-30T12:45:20.377742+0000","last_fullsized":"2023-10-30T12:45:20.377742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:13.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084274+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:24.084274+0000","last_peered":"2023-10-30T12:45:24.084274+0000","last_clean":"2023-10-30T12:45:24.084274+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:24.084274+0000","last_undegraded":"2023-10-30T12:45:24.084274+0000","last_fullsized":"2023-10-30T12:45:24.084274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377958+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:20.377958+0000","last_peered":"2023-10-30T12:45:20.377958+0000","last_clean":"2023-10-30T12:45:20.377958+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:20.377958+0000","last_undegraded":"2023-10-30T12:45:20.377958+0000","last_fullsized":"2023-10-30T12:45:20.377958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:08.137679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.207109+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:35.207109+0000","last_peered":"2023-10-30T12:45:35.207109+0000","last_clean":"2023-10-30T12:45:35.207109+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:35.207109+0000","last_undegraded":"2023-10-30T12:45:35.207109+0000","last_fullsized":"2023-10-30T12:45:35.207109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471292+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:20.471292+0000","last_peered":"2023-10-30T12:45:20.471292+0000","last_clean":"2023-10-30T12:45:20.471292+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:20.471292+0000","last_undegraded":"2023-10-30T12:45:20.471292+0000","last_fullsized":"2023-10-30T12:45:20.471292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:26.117300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.078665+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:24.078665+0000","last_peered":"2023-10-30T12:45:24.078665+0000","last_clean":"2023-10-30T12:45:24.078665+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:24.078665+0000","last_undegraded":"2023-10-30T12:45:24.078665+0000","last_fullsized":"2023-10-30T12:45:24.078665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471526+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:20.471526+0000","last_peered":"2023-10-30T12:45:20.471526+0000","last_clean":"2023-10-30T12:45:20.471526+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:20.471526+0000","last_undegraded":"2023-10-30T12:45:20.471526+0000","last_fullsized":"2023-10-30T12:45:20.471526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:57:18.999410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'9","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.208649+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:35.208649+0000","last_peered":"2023-10-30T12:45:35.208649+0000","last_clean":"2023-10-30T12:45:35.208649+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:35.208649+0000","last_undegraded":"2023-10-30T12:45:35.208649+0000","last_fullsized":"2023-10-30T12:45:35.208649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":819200,"data_stored":662582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":73704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":72276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":221184,"data_stored":71870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":74295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":71716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":71842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":660907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:48.200 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:48.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 Processor -- start 2023-10-30T12:45:48.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 -- start start 2023-10-30T12:45:48.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48ec0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0c5030 con 0x7f48ec151c20 2023-10-30T12:45:48.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec157830 con 0x7f48ec151840 2023-10-30T12:45:48.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f3e09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec157b10 con 0x7f48ec06bd10 2023-10-30T12:45:48.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f13a4640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f48ec151c20 0x7f48ec06b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60528/0 (socket says 172.21.15.8:60528) 2023-10-30T12:45:48.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f13a4640 1 -- 172.21.15.8:0/1160862850 learned_addr learned my addr 172.21.15.8:0/1160862850 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:48.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f23a6640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48ec0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918583422 0 0) 0x7f48ec0c5030 con 0x7f48ec151c20 2023-10-30T12:45:48.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.381+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d8003530 con 0x7f48ec151c20 2023-10-30T12:45:48.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1299211438 0 0) 0x7f48d8003530 con 0x7f48ec151c20 2023-10-30T12:45:48.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 msgr2=0x7f48ec0c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:48.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48ec0c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 >> v1:172.21.15.46:6789/0 conn(0x7f48ec151840 legacy=0x7f48ec072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec157df0 con 0x7f48ec151c20 2023-10-30T12:45:48.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f48dc001680 con 0x7f48ec151c20 2023-10-30T12:45:48.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 >> v1:172.21.15.8:6789/0 conn(0x7f48ec151c20 legacy=0x7f48ec06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d80039b0 0x7f48d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48d80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48d8004d10 0x7f48d8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec157b10 con 0x7f48d8004d10 2023-10-30T12:45:48.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d800b990 con 0x7f48d80039b0 2023-10-30T12:45:48.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d800bb50 con 0x7f48ec06bd10 2023-10-30T12:45:48.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48d8004d10 0x7f48d8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f1ba5640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d80039b0 0x7f48d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f13a4640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48d80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 -- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 msgr2=0x7f48d80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:48.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec06bd10 0x7f48d80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 -- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d80039b0 msgr2=0x7f48d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:48.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d80039b0 0x7f48d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 -- 172.21.15.8:0/1160862850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8003530 con 0x7f48d8004d10 2023-10-30T12:45:48.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f1ba5640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d80039b0 0x7f48d8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:48.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.385+0000 7f48f23a6640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48d8004d10 0x7f48d8005190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fc0 tx=0x7f48e401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377866c2a576b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40a5c70 con 0x7f48d8004d10 2023-10-30T12:45:48.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e400f040 con 0x7f48d8004d10 2023-10-30T12:45:48.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f0ba3640 1 -- 172.21.15.8:0/1160862850 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40ad440 con 0x7f48d8004d10 2023-10-30T12:45:48.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- 172.21.15.8:0/1160862850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48d8004d10 msgr2=0x7f48d8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 --2- 172.21.15.8:0/1160862850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48d8004d10 0x7f48d8005190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fc0 tx=0x7f48e401cc60 comp rx=0 tx=0).stop 2023-10-30T12:45:48.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f23a6640 1 -- 172.21.15.8:0/1160862850 reap_dead start 2023-10-30T12:45:48.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- 172.21.15.8:0/1160862850 shutdown_connections 2023-10-30T12:45:48.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- 172.21.15.8:0/1160862850 >> 172.21.15.8:0/1160862850 conn(0x7f48ec0cac70 msgr2=0x7f48ec0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- 172.21.15.8:0/1160862850 shutdown_connections 2023-10-30T12:45:48.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- 172.21.15.8:0/1160862850 wait complete. 2023-10-30T12:45:48.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 Processor -- start 2023-10-30T12:45:48.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- start start 2023-10-30T12:45:48.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec144de0 0x7f48ec1eab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec1451c0 0x7f48ec156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0712b0 con 0x7f48ec06fdf0 2023-10-30T12:45:48.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec1ea120 con 0x7f48ec144de0 2023-10-30T12:45:48.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f3e09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec071f70 con 0x7f48ec1451c0 2023-10-30T12:45:48.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f13a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f13a4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53464/0 (socket says 172.21.15.8:53464) 2023-10-30T12:45:48.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.389+0000 7f48f13a4640 1 -- 172.21.15.8:0/472489402 learned_addr learned my addr 172.21.15.8:0/472489402 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:48.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f1ba5640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec144de0 0x7f48ec1eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f23a6640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec1451c0 0x7f48ec156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 -- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec1451c0 msgr2=0x7f48ec156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec1451c0 0x7f48ec156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 -- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec144de0 msgr2=0x7f48ec1eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec144de0 0x7f48ec1eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 -- 172.21.15.8:0/472489402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec1ee0a0 con 0x7f48ec06fdf0 2023-10-30T12:45:48.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f13a4640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e920 tx=0x7f48dc00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00c930 con 0x7f48ec06fdf0 2023-10-30T12:45:48.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00cab0 con 0x7f48ec06fdf0 2023-10-30T12:45:48.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec1eb070 con 0x7f48ec06fdf0 2023-10-30T12:45:48.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00ce60 con 0x7f48ec06fdf0 2023-10-30T12:45:48.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec1ee340 con 0x7f48ec06fdf0 2023-10-30T12:45:48.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec1ee530 con 0x7f48ec06fdf0 2023-10-30T12:45:48.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.393+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48dc013420 con 0x7f48ec06fdf0 2023-10-30T12:45:48.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.397+0000 7f48e27fc640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 0x7f48d0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.397+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48dc083cc0 con 0x7f48ec06fdf0 2023-10-30T12:45:48.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.397+0000 7f48f1ba5640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 0x7f48d0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.397+0000 7f48f1ba5640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 0x7f48d0068990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48e8005580 tx=0x7f48e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.401+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48dc08a080 con 0x7f48ec06fdf0 2023-10-30T12:45:48.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.553+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48ec1583b0 con 0x7f48d00664d0 2023-10-30T12:45:48.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.561+0000 7f48e27fc640 1 -- 172.21.15.8:0/472489402 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264634 (secure 0 0 0) 0x7f48e8001480 con 0x7f48d00664d0 2023-10-30T12:45:48.568 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:48.570 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:48.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 msgr2=0x7f48d0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 0x7f48d0068990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48e8005580 tx=0x7f48e8009040 comp rx=0 tx=0).stop 2023-10-30T12:45:48.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 msgr2=0x7f48ec1569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e920 tx=0x7f48dc00ede0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 shutdown_connections 2023-10-30T12:45:48.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48d00664d0 0x7f48d0068990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48ec1451c0 0x7f48ec156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48ec06fdf0 0x7f48ec1569c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 --2- 172.21.15.8:0/472489402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48ec144de0 0x7f48ec1eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 >> 172.21.15.8:0/472489402 conn(0x7f48ec0cac70 msgr2=0x7f48ec0c6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 shutdown_connections 2023-10-30T12:45:48.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.565+0000 7f48f3e09640 1 -- 172.21.15.8:0/472489402 wait complete. 2023-10-30T12:45:48.588 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-30T12:45:47.618606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252324,"kb_used_data":2916,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722396,"statfs":{"total":773094113280,"available":772835733504,"internally_reserved":0,"allocated":2985984,"data_stored":1759192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071955+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:20.071955+0000","last_peered":"2023-10-30T12:45:20.071955+0000","last_clean":"2023-10-30T12:45:20.071955+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:20.071955+0000","last_undegraded":"2023-10-30T12:45:20.071955+0000","last_fullsized":"2023-10-30T12:45:20.071955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:33:48.910689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067349+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:20.067349+0000","last_peered":"2023-10-30T12:45:20.067349+0000","last_clean":"2023-10-30T12:45:20.067349+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:20.067349+0000","last_undegraded":"2023-10-30T12:45:20.067349+0000","last_fullsized":"2023-10-30T12:45:20.067349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:34.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470964+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:20.470964+0000","last_peered":"2023-10-30T12:45:20.470964+0000","last_clean":"2023-10-30T12:45:20.470964+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:20.470964+0000","last_undegraded":"2023-10-30T12:45:20.470964+0000","last_fullsized":"2023-10-30T12:45:20.470964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:45:42.325441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377353+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:20.377353+0000","last_peered":"2023-10-30T12:45:20.377353+0000","last_clean":"2023-10-30T12:45:20.377353+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:20.377353+0000","last_undegraded":"2023-10-30T12:45:20.377353+0000","last_fullsized":"2023-10-30T12:45:20.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:59.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069882+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:20.069882+0000","last_peered":"2023-10-30T12:45:20.069882+0000","last_clean":"2023-10-30T12:45:20.069882+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:20.069882+0000","last_undegraded":"2023-10-30T12:45:20.069882+0000","last_fullsized":"2023-10-30T12:45:20.069882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:09.263782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376997+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:20.376997+0000","last_peered":"2023-10-30T12:45:20.376997+0000","last_clean":"2023-10-30T12:45:20.376997+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:20.376997+0000","last_undegraded":"2023-10-30T12:45:20.376997+0000","last_fullsized":"2023-10-30T12:45:20.376997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:48:04.677113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069348+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:20.069348+0000","last_peered":"2023-10-30T12:45:20.069348+0000","last_clean":"2023-10-30T12:45:20.069348+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:20.069348+0000","last_undegraded":"2023-10-30T12:45:20.069348+0000","last_fullsized":"2023-10-30T12:45:20.069348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:43:52.826848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470720+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:20.470720+0000","last_peered":"2023-10-30T12:45:20.470720+0000","last_clean":"2023-10-30T12:45:20.470720+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:20.470720+0000","last_undegraded":"2023-10-30T12:45:20.470720+0000","last_fullsized":"2023-10-30T12:45:20.470720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:27:17.979749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072297+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:20.072297+0000","last_peered":"2023-10-30T12:45:20.072297+0000","last_clean":"2023-10-30T12:45:20.072297+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:20.072297+0000","last_undegraded":"2023-10-30T12:45:20.072297+0000","last_fullsized":"2023-10-30T12:45:20.072297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:28.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471673+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:20.471673+0000","last_peered":"2023-10-30T12:45:20.471673+0000","last_clean":"2023-10-30T12:45:20.471673+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:20.471673+0000","last_undegraded":"2023-10-30T12:45:20.471673+0000","last_fullsized":"2023-10-30T12:45:20.471673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:06:39.736671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471336+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:20.471336+0000","last_peered":"2023-10-30T12:45:20.471336+0000","last_clean":"2023-10-30T12:45:20.471336+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:20.471336+0000","last_undegraded":"2023-10-30T12:45:20.471336+0000","last_fullsized":"2023-10-30T12:45:20.471336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:55.158597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071808+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:20.071808+0000","last_peered":"2023-10-30T12:45:20.071808+0000","last_clean":"2023-10-30T12:45:20.071808+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:20.071808+0000","last_undegraded":"2023-10-30T12:45:20.071808+0000","last_fullsized":"2023-10-30T12:45:20.071808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:18:07.085147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470864+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:20.470864+0000","last_peered":"2023-10-30T12:45:20.470864+0000","last_clean":"2023-10-30T12:45:20.470864+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:20.470864+0000","last_undegraded":"2023-10-30T12:45:20.470864+0000","last_fullsized":"2023-10-30T12:45:20.470864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:45:04.008631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470638+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:20.470638+0000","last_peered":"2023-10-30T12:45:20.470638+0000","last_clean":"2023-10-30T12:45:20.470638+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:20.470638+0000","last_undegraded":"2023-10-30T12:45:20.470638+0000","last_fullsized":"2023-10-30T12:45:20.470638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:02:04.825098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:23.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376825+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:20.376825+0000","last_peered":"2023-10-30T12:45:20.376825+0000","last_clean":"2023-10-30T12:45:20.376825+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:20.376825+0000","last_undegraded":"2023-10-30T12:45:20.376825+0000","last_fullsized":"2023-10-30T12:45:20.376825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:48:16.344709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377969+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:20.377969+0000","last_peered":"2023-10-30T12:45:20.377969+0000","last_clean":"2023-10-30T12:45:20.377969+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:20.377969+0000","last_undegraded":"2023-10-30T12:45:20.377969+0000","last_fullsized":"2023-10-30T12:45:20.377969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:08.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471127+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:20.471127+0000","last_peered":"2023-10-30T12:45:20.471127+0000","last_clean":"2023-10-30T12:45:20.471127+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:20.471127+0000","last_undegraded":"2023-10-30T12:45:20.471127+0000","last_fullsized":"2023-10-30T12:45:20.471127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:05.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070768+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:20.070768+0000","last_peered":"2023-10-30T12:45:20.070768+0000","last_clean":"2023-10-30T12:45:20.070768+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:20.070768+0000","last_undegraded":"2023-10-30T12:45:20.070768+0000","last_fullsized":"2023-10-30T12:45:20.070768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:17.483158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471411+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:20.471411+0000","last_peered":"2023-10-30T12:45:20.471411+0000","last_clean":"2023-10-30T12:45:20.471411+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:20.471411+0000","last_undegraded":"2023-10-30T12:45:20.471411+0000","last_fullsized":"2023-10-30T12:45:20.471411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:16:21.392983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470962+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:20.470962+0000","last_peered":"2023-10-30T12:45:20.470962+0000","last_clean":"2023-10-30T12:45:20.470962+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:20.470962+0000","last_undegraded":"2023-10-30T12:45:20.470962+0000","last_fullsized":"2023-10-30T12:45:20.470962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:02.564276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377142+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:20.377142+0000","last_peered":"2023-10-30T12:45:20.377142+0000","last_clean":"2023-10-30T12:45:20.377142+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:20.377142+0000","last_undegraded":"2023-10-30T12:45:20.377142+0000","last_fullsized":"2023-10-30T12:45:20.377142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:33:03.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470206+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:20.470206+0000","last_peered":"2023-10-30T12:45:20.470206+0000","last_clean":"2023-10-30T12:45:20.470206+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:20.470206+0000","last_undegraded":"2023-10-30T12:45:20.470206+0000","last_fullsized":"2023-10-30T12:45:20.470206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:31:49.660950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376996+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:20.376996+0000","last_peered":"2023-10-30T12:45:20.376996+0000","last_clean":"2023-10-30T12:45:20.376996+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:20.376996+0000","last_undegraded":"2023-10-30T12:45:20.376996+0000","last_fullsized":"2023-10-30T12:45:20.376996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:51.286544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471712+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:20.471712+0000","last_peered":"2023-10-30T12:45:20.471712+0000","last_clean":"2023-10-30T12:45:20.471712+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:20.471712+0000","last_undegraded":"2023-10-30T12:45:20.471712+0000","last_fullsized":"2023-10-30T12:45:20.471712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:57.636148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471671+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:20.471671+0000","last_peered":"2023-10-30T12:45:20.471671+0000","last_clean":"2023-10-30T12:45:20.471671+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:20.471671+0000","last_undegraded":"2023-10-30T12:45:20.471671+0000","last_fullsized":"2023-10-30T12:45:20.471671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:30.410457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067039+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:20.067039+0000","last_peered":"2023-10-30T12:45:20.067039+0000","last_clean":"2023-10-30T12:45:20.067039+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:20.067039+0000","last_undegraded":"2023-10-30T12:45:20.067039+0000","last_fullsized":"2023-10-30T12:45:20.067039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:46.788124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471494+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:20.471494+0000","last_peered":"2023-10-30T12:45:20.471494+0000","last_clean":"2023-10-30T12:45:20.471494+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:20.471494+0000","last_undegraded":"2023-10-30T12:45:20.471494+0000","last_fullsized":"2023-10-30T12:45:20.471494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:44.555138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471084+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:20.471084+0000","last_peered":"2023-10-30T12:45:20.471084+0000","last_clean":"2023-10-30T12:45:20.471084+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:20.471084+0000","last_undegraded":"2023-10-30T12:45:20.471084+0000","last_fullsized":"2023-10-30T12:45:20.471084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:29.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377430+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:20.377430+0000","last_peered":"2023-10-30T12:45:20.377430+0000","last_clean":"2023-10-30T12:45:20.377430+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:20.377430+0000","last_undegraded":"2023-10-30T12:45:20.377430+0000","last_fullsized":"2023-10-30T12:45:20.377430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:50:53.563609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471353+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:20.471353+0000","last_peered":"2023-10-30T12:45:20.471353+0000","last_clean":"2023-10-30T12:45:20.471353+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:20.471353+0000","last_undegraded":"2023-10-30T12:45:20.471353+0000","last_fullsized":"2023-10-30T12:45:20.471353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:12:59.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470912+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:20.470912+0000","last_peered":"2023-10-30T12:45:20.470912+0000","last_clean":"2023-10-30T12:45:20.470912+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:20.470912+0000","last_undegraded":"2023-10-30T12:45:20.470912+0000","last_fullsized":"2023-10-30T12:45:20.470912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:10.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071561+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:20.071561+0000","last_peered":"2023-10-30T12:45:20.071561+0000","last_clean":"2023-10-30T12:45:20.071561+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:20.071561+0000","last_undegraded":"2023-10-30T12:45:20.071561+0000","last_fullsized":"2023-10-30T12:45:20.071561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:17:56.183441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.167894+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:29.167894+0000","last_peered":"2023-10-30T12:45:29.167894+0000","last_clean":"2023-10-30T12:45:29.167894+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:29.167894+0000","last_undegraded":"2023-10-30T12:45:29.167894+0000","last_fullsized":"2023-10-30T12:45:29.167894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471141+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:20.471141+0000","last_peered":"2023-10-30T12:45:20.471141+0000","last_clean":"2023-10-30T12:45:20.471141+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:20.471141+0000","last_undegraded":"2023-10-30T12:45:20.471141+0000","last_fullsized":"2023-10-30T12:45:20.471141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:23:00.831740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203892+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:35.203892+0000","last_peered":"2023-10-30T12:45:35.203892+0000","last_clean":"2023-10-30T12:45:35.203892+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:35.203892+0000","last_undegraded":"2023-10-30T12:45:35.203892+0000","last_fullsized":"2023-10-30T12:45:35.203892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203844+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:35.203844+0000","last_peered":"2023-10-30T12:45:35.203844+0000","last_clean":"2023-10-30T12:45:35.203844+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:35.203844+0000","last_undegraded":"2023-10-30T12:45:35.203844+0000","last_fullsized":"2023-10-30T12:45:35.203844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470208+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:20.470208+0000","last_peered":"2023-10-30T12:45:20.470208+0000","last_clean":"2023-10-30T12:45:20.470208+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:20.470208+0000","last_undegraded":"2023-10-30T12:45:20.470208+0000","last_fullsized":"2023-10-30T12:45:20.470208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:26.174484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377740+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:20.377740+0000","last_peered":"2023-10-30T12:45:20.377740+0000","last_clean":"2023-10-30T12:45:20.377740+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:20.377740+0000","last_undegraded":"2023-10-30T12:45:20.377740+0000","last_fullsized":"2023-10-30T12:45:20.377740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:12:12.444137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157838+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:29.157838+0000","last_peered":"2023-10-30T12:45:29.157838+0000","last_clean":"2023-10-30T12:45:29.157838+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:29.157838+0000","last_undegraded":"2023-10-30T12:45:29.157838+0000","last_fullsized":"2023-10-30T12:45:29.157838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377703+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:20.377703+0000","last_peered":"2023-10-30T12:45:20.377703+0000","last_clean":"2023-10-30T12:45:20.377703+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:20.377703+0000","last_undegraded":"2023-10-30T12:45:20.377703+0000","last_fullsized":"2023-10-30T12:45:20.377703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:40:41.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.158692+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:29.158692+0000","last_peered":"2023-10-30T12:45:29.158692+0000","last_clean":"2023-10-30T12:45:29.158692+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:29.158692+0000","last_undegraded":"2023-10-30T12:45:29.158692+0000","last_fullsized":"2023-10-30T12:45:29.158692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471307+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:20.471307+0000","last_peered":"2023-10-30T12:45:20.471307+0000","last_clean":"2023-10-30T12:45:20.471307+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:20.471307+0000","last_undegraded":"2023-10-30T12:45:20.471307+0000","last_fullsized":"2023-10-30T12:45:20.471307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:07:09.393386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.086171+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:24.086171+0000","last_peered":"2023-10-30T12:45:24.086171+0000","last_clean":"2023-10-30T12:45:24.086171+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:24.086171+0000","last_undegraded":"2023-10-30T12:45:24.086171+0000","last_fullsized":"2023-10-30T12:45:24.086171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072148+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:20.072148+0000","last_peered":"2023-10-30T12:45:20.072148+0000","last_clean":"2023-10-30T12:45:20.072148+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:20.072148+0000","last_undegraded":"2023-10-30T12:45:20.072148+0000","last_fullsized":"2023-10-30T12:45:20.072148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:17.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'11","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:43.496620+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:43.496620+0000","last_peered":"2023-10-30T12:45:43.496620+0000","last_clean":"2023-10-30T12:45:43.496620+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:43.496620+0000","last_undegraded":"2023-10-30T12:45:43.496620+0000","last_fullsized":"2023-10-30T12:45:43.496620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069876+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:20.069876+0000","last_peered":"2023-10-30T12:45:20.069876+0000","last_clean":"2023-10-30T12:45:20.069876+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:20.069876+0000","last_undegraded":"2023-10-30T12:45:20.069876+0000","last_fullsized":"2023-10-30T12:45:20.069876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:07:42.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201182+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:35.201182+0000","last_peered":"2023-10-30T12:45:35.201182+0000","last_clean":"2023-10-30T12:45:35.201182+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:35.201182+0000","last_undegraded":"2023-10-30T12:45:35.201182+0000","last_fullsized":"2023-10-30T12:45:35.201182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470757+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:20.470757+0000","last_peered":"2023-10-30T12:45:20.470757+0000","last_clean":"2023-10-30T12:45:20.470757+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:20.470757+0000","last_undegraded":"2023-10-30T12:45:20.470757+0000","last_fullsized":"2023-10-30T12:45:20.470757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:17.831426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.164676+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:29.164676+0000","last_peered":"2023-10-30T12:45:29.164676+0000","last_clean":"2023-10-30T12:45:29.164676+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:29.164676+0000","last_undegraded":"2023-10-30T12:45:29.164676+0000","last_fullsized":"2023-10-30T12:45:29.164676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070431+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:20.070431+0000","last_peered":"2023-10-30T12:45:20.070431+0000","last_clean":"2023-10-30T12:45:20.070431+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:20.070431+0000","last_undegraded":"2023-10-30T12:45:20.070431+0000","last_fullsized":"2023-10-30T12:45:20.070431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:14:31.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.210377+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:35.210377+0000","last_peered":"2023-10-30T12:45:35.210377+0000","last_clean":"2023-10-30T12:45:35.210377+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:35.210377+0000","last_undegraded":"2023-10-30T12:45:35.210377+0000","last_fullsized":"2023-10-30T12:45:35.210377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470957+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:20.470957+0000","last_peered":"2023-10-30T12:45:20.470957+0000","last_clean":"2023-10-30T12:45:20.470957+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:20.470957+0000","last_undegraded":"2023-10-30T12:45:20.470957+0000","last_fullsized":"2023-10-30T12:45:20.470957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:13.282854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'15","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201839+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:35.201839+0000","last_peered":"2023-10-30T12:45:35.201839+0000","last_clean":"2023-10-30T12:45:35.201839+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:35.201839+0000","last_undegraded":"2023-10-30T12:45:35.201839+0000","last_fullsized":"2023-10-30T12:45:35.201839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069448+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:20.069448+0000","last_peered":"2023-10-30T12:45:20.069448+0000","last_clean":"2023-10-30T12:45:20.069448+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:20.069448+0000","last_undegraded":"2023-10-30T12:45:20.069448+0000","last_fullsized":"2023-10-30T12:45:20.069448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:14.563685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.079122+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:24.079122+0000","last_peered":"2023-10-30T12:45:24.079122+0000","last_clean":"2023-10-30T12:45:24.079122+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:24.079122+0000","last_undegraded":"2023-10-30T12:45:24.079122+0000","last_fullsized":"2023-10-30T12:45:24.079122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071276+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:20.071276+0000","last_peered":"2023-10-30T12:45:20.071276+0000","last_clean":"2023-10-30T12:45:20.071276+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:20.071276+0000","last_undegraded":"2023-10-30T12:45:20.071276+0000","last_fullsized":"2023-10-30T12:45:20.071276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:43.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.077908+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:24.077908+0000","last_peered":"2023-10-30T12:45:24.077908+0000","last_clean":"2023-10-30T12:45:24.077908+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:24.077908+0000","last_undegraded":"2023-10-30T12:45:24.077908+0000","last_fullsized":"2023-10-30T12:45:24.077908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072543+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:20.072543+0000","last_peered":"2023-10-30T12:45:20.072543+0000","last_clean":"2023-10-30T12:45:20.072543+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:20.072543+0000","last_undegraded":"2023-10-30T12:45:20.072543+0000","last_fullsized":"2023-10-30T12:45:20.072543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:54.641020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067271+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:20.067271+0000","last_peered":"2023-10-30T12:45:20.067271+0000","last_clean":"2023-10-30T12:45:20.067271+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:20.067271+0000","last_undegraded":"2023-10-30T12:45:20.067271+0000","last_fullsized":"2023-10-30T12:45:20.067271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378262+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:20.378262+0000","last_peered":"2023-10-30T12:45:20.378262+0000","last_clean":"2023-10-30T12:45:20.378262+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:20.378262+0000","last_undegraded":"2023-10-30T12:45:20.378262+0000","last_fullsized":"2023-10-30T12:45:20.378262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:08.395587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.205713+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:35.205713+0000","last_peered":"2023-10-30T12:45:35.205713+0000","last_clean":"2023-10-30T12:45:35.205713+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:35.205713+0000","last_undegraded":"2023-10-30T12:45:35.205713+0000","last_fullsized":"2023-10-30T12:45:35.205713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069897+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:20.069897+0000","last_peered":"2023-10-30T12:45:20.069897+0000","last_clean":"2023-10-30T12:45:20.069897+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:20.069897+0000","last_undegraded":"2023-10-30T12:45:20.069897+0000","last_fullsized":"2023-10-30T12:45:20.069897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:58:00.569432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071157+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:19.071157+0000","last_peered":"2023-10-30T12:45:19.071157+0000","last_clean":"2023-10-30T12:45:19.071157+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:19.071157+0000","last_undegraded":"2023-10-30T12:45:19.071157+0000","last_fullsized":"2023-10-30T12:45:19.071157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156996+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:29.156996+0000","last_peered":"2023-10-30T12:45:29.156996+0000","last_clean":"2023-10-30T12:45:29.156996+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:29.156996+0000","last_undegraded":"2023-10-30T12:45:29.156996+0000","last_fullsized":"2023-10-30T12:45:29.156996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470630+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:20.470630+0000","last_peered":"2023-10-30T12:45:20.470630+0000","last_clean":"2023-10-30T12:45:20.470630+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:20.470630+0000","last_undegraded":"2023-10-30T12:45:20.470630+0000","last_fullsized":"2023-10-30T12:45:20.470630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:25:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201698+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:35.201698+0000","last_peered":"2023-10-30T12:45:35.201698+0000","last_clean":"2023-10-30T12:45:35.201698+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:35.201698+0000","last_undegraded":"2023-10-30T12:45:35.201698+0000","last_fullsized":"2023-10-30T12:45:35.201698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470604+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:20.470604+0000","last_peered":"2023-10-30T12:45:20.470604+0000","last_clean":"2023-10-30T12:45:20.470604+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:20.470604+0000","last_undegraded":"2023-10-30T12:45:20.470604+0000","last_fullsized":"2023-10-30T12:45:20.470604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:55.822788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.202222+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:35.202222+0000","last_peered":"2023-10-30T12:45:35.202222+0000","last_clean":"2023-10-30T12:45:35.202222+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:35.202222+0000","last_undegraded":"2023-10-30T12:45:35.202222+0000","last_fullsized":"2023-10-30T12:45:35.202222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066971+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:20.066971+0000","last_peered":"2023-10-30T12:45:20.066971+0000","last_clean":"2023-10-30T12:45:20.066971+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:20.066971+0000","last_undegraded":"2023-10-30T12:45:20.066971+0000","last_fullsized":"2023-10-30T12:45:20.066971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:17:16.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377841+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:20.377841+0000","last_peered":"2023-10-30T12:45:20.377841+0000","last_clean":"2023-10-30T12:45:20.377841+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:20.377841+0000","last_undegraded":"2023-10-30T12:45:20.377841+0000","last_fullsized":"2023-10-30T12:45:20.377841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072149+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:20.072149+0000","last_peered":"2023-10-30T12:45:20.072149+0000","last_clean":"2023-10-30T12:45:20.072149+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:20.072149+0000","last_undegraded":"2023-10-30T12:45:20.072149+0000","last_fullsized":"2023-10-30T12:45:20.072149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630601+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.630601+0000","last_peered":"2023-10-30T12:45:19.630601+0000","last_clean":"2023-10-30T12:45:19.630601+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.630601+0000","last_undegraded":"2023-10-30T12:45:19.630601+0000","last_fullsized":"2023-10-30T12:45:19.630601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378093+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:20.378093+0000","last_peered":"2023-10-30T12:45:20.378093+0000","last_clean":"2023-10-30T12:45:20.378093+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:20.378093+0000","last_undegraded":"2023-10-30T12:45:20.378093+0000","last_fullsized":"2023-10-30T12:45:20.378093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:05.357577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071109+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:20.071109+0000","last_peered":"2023-10-30T12:45:20.071109+0000","last_clean":"2023-10-30T12:45:20.071109+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:20.071109+0000","last_undegraded":"2023-10-30T12:45:20.071109+0000","last_fullsized":"2023-10-30T12:45:20.071109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066974+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:20.066974+0000","last_peered":"2023-10-30T12:45:20.066974+0000","last_clean":"2023-10-30T12:45:20.066974+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:20.066974+0000","last_undegraded":"2023-10-30T12:45:20.066974+0000","last_fullsized":"2023-10-30T12:45:20.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:46:43.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201094+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:35.201094+0000","last_peered":"2023-10-30T12:45:35.201094+0000","last_clean":"2023-10-30T12:45:35.201094+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:35.201094+0000","last_undegraded":"2023-10-30T12:45:35.201094+0000","last_fullsized":"2023-10-30T12:45:35.201094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471700+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:20.471700+0000","last_peered":"2023-10-30T12:45:20.471700+0000","last_clean":"2023-10-30T12:45:20.471700+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:20.471700+0000","last_undegraded":"2023-10-30T12:45:20.471700+0000","last_fullsized":"2023-10-30T12:45:20.471700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:25.320280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157353+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:29.157353+0000","last_peered":"2023-10-30T12:45:29.157353+0000","last_clean":"2023-10-30T12:45:29.157353+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:29.157353+0000","last_undegraded":"2023-10-30T12:45:29.157353+0000","last_fullsized":"2023-10-30T12:45:29.157353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471488+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:20.471488+0000","last_peered":"2023-10-30T12:45:20.471488+0000","last_clean":"2023-10-30T12:45:20.471488+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:20.471488+0000","last_undegraded":"2023-10-30T12:45:20.471488+0000","last_fullsized":"2023-10-30T12:45:20.471488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:14.407746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156619+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:29.156619+0000","last_peered":"2023-10-30T12:45:29.156619+0000","last_clean":"2023-10-30T12:45:29.156619+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:29.156619+0000","last_undegraded":"2023-10-30T12:45:29.156619+0000","last_fullsized":"2023-10-30T12:45:29.156619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071773+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:20.071773+0000","last_peered":"2023-10-30T12:45:20.071773+0000","last_clean":"2023-10-30T12:45:20.071773+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:20.071773+0000","last_undegraded":"2023-10-30T12:45:20.071773+0000","last_fullsized":"2023-10-30T12:45:20.071773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:09.037055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.165872+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:29.165872+0000","last_peered":"2023-10-30T12:45:29.165872+0000","last_clean":"2023-10-30T12:45:29.165872+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:29.165872+0000","last_undegraded":"2023-10-30T12:45:29.165872+0000","last_fullsized":"2023-10-30T12:45:29.165872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470363+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:20.470363+0000","last_peered":"2023-10-30T12:45:20.470363+0000","last_clean":"2023-10-30T12:45:20.470363+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:20.470363+0000","last_undegraded":"2023-10-30T12:45:20.470363+0000","last_fullsized":"2023-10-30T12:45:20.470363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:45:36.069824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067111+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:20.067111+0000","last_peered":"2023-10-30T12:45:20.067111+0000","last_clean":"2023-10-30T12:45:20.067111+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:20.067111+0000","last_undegraded":"2023-10-30T12:45:20.067111+0000","last_fullsized":"2023-10-30T12:45:20.067111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:21.343385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.151892+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:29.151892+0000","last_peered":"2023-10-30T12:45:29.151892+0000","last_clean":"2023-10-30T12:45:29.151892+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:29.151892+0000","last_undegraded":"2023-10-30T12:45:29.151892+0000","last_fullsized":"2023-10-30T12:45:29.151892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470979+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:20.470979+0000","last_peered":"2023-10-30T12:45:20.470979+0000","last_clean":"2023-10-30T12:45:20.470979+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:20.470979+0000","last_undegraded":"2023-10-30T12:45:20.470979+0000","last_fullsized":"2023-10-30T12:45:20.470979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:44.769458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377742+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:20.377742+0000","last_peered":"2023-10-30T12:45:20.377742+0000","last_clean":"2023-10-30T12:45:20.377742+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:20.377742+0000","last_undegraded":"2023-10-30T12:45:20.377742+0000","last_fullsized":"2023-10-30T12:45:20.377742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:13.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084274+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:24.084274+0000","last_peered":"2023-10-30T12:45:24.084274+0000","last_clean":"2023-10-30T12:45:24.084274+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:24.084274+0000","last_undegraded":"2023-10-30T12:45:24.084274+0000","last_fullsized":"2023-10-30T12:45:24.084274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377958+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:20.377958+0000","last_peered":"2023-10-30T12:45:20.377958+0000","last_clean":"2023-10-30T12:45:20.377958+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:20.377958+0000","last_undegraded":"2023-10-30T12:45:20.377958+0000","last_fullsized":"2023-10-30T12:45:20.377958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:08.137679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.207109+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:35.207109+0000","last_peered":"2023-10-30T12:45:35.207109+0000","last_clean":"2023-10-30T12:45:35.207109+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:35.207109+0000","last_undegraded":"2023-10-30T12:45:35.207109+0000","last_fullsized":"2023-10-30T12:45:35.207109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471292+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:20.471292+0000","last_peered":"2023-10-30T12:45:20.471292+0000","last_clean":"2023-10-30T12:45:20.471292+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:20.471292+0000","last_undegraded":"2023-10-30T12:45:20.471292+0000","last_fullsized":"2023-10-30T12:45:20.471292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:26.117300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.078665+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:24.078665+0000","last_peered":"2023-10-30T12:45:24.078665+0000","last_clean":"2023-10-30T12:45:24.078665+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:24.078665+0000","last_undegraded":"2023-10-30T12:45:24.078665+0000","last_fullsized":"2023-10-30T12:45:24.078665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471526+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:20.471526+0000","last_peered":"2023-10-30T12:45:20.471526+0000","last_clean":"2023-10-30T12:45:20.471526+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:20.471526+0000","last_undegraded":"2023-10-30T12:45:20.471526+0000","last_fullsized":"2023-10-30T12:45:20.471526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:57:18.999410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'9","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.208649+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:35.208649+0000","last_peered":"2023-10-30T12:45:35.208649+0000","last_clean":"2023-10-30T12:45:35.208649+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:35.208649+0000","last_undegraded":"2023-10-30T12:45:35.208649+0000","last_fullsized":"2023-10-30T12:45:35.208649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":819200,"data_stored":662582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":73704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":72276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":221184,"data_stored":71870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":74295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":71716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":71842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":660907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:48.593 INFO:tasks.mds_thrash.ceph_manager:clean! 2023-10-30T12:45:48.593 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:45:48.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 Processor -- start 2023-10-30T12:45:48.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 -- start start 2023-10-30T12:45:48.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2b300c4a10 0x7f2b3015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 0x7f2b300ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b30159f20 con 0x7f2b30153c80 2023-10-30T12:45:48.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3015dec0 con 0x7f2b300c4a10 2023-10-30T12:45:48.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b3593e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b301599a0 con 0x7f2b300c42a0 2023-10-30T12:45:48.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b2effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2b30153c80 0x7f2b3015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60536/0 (socket says 172.21.15.8:60536) 2023-10-30T12:45:48.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b2effd640 1 -- 172.21.15.8:0/4047863792 learned_addr learned my addr 172.21.15.8:0/4047863792 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:48.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b27fff640 1 --2- 172.21.15.8:0/4047863792 >> v2:172.21.15.8:3300/0 conn(0x7f2b300c4a10 0x7f2b3015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b2f7fe640 1 --2- 172.21.15.8:0/4047863792 >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 0x7f2b300ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b27fff640 1 -- 172.21.15.8:0/4047863792 >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 msgr2=0x7f2b300ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:48.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b27fff640 1 --2- 172.21.15.8:0/4047863792 >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 0x7f2b300ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b27fff640 1 -- 172.21.15.8:0/4047863792 >> v1:172.21.15.8:6789/0 conn(0x7f2b30153c80 legacy=0x7f2b3015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b27fff640 1 -- 172.21.15.8:0/4047863792 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3015d620 con 0x7f2b300c4a10 2023-10-30T12:45:48.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.773+0000 7f2b2f7fe640 1 --2- 172.21.15.8:0/4047863792 >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 0x7f2b300ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:48.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b27fff640 1 --2- 172.21.15.8:0/4047863792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b300c4a10 0x7f2b3015cf90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b1d40 tx=0x7f2b1801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaf76fdc89e1934 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/4047863792 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180a8950 con 0x7f2b300c4a10 2023-10-30T12:45:48.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/4047863792 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b180a8ab0 con 0x7f2b300c4a10 2023-10-30T12:45:48.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/4047863792 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180a8e40 con 0x7f2b300c4a10 2023-10-30T12:45:48.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 -- 172.21.15.8:0/4047863792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b300c4a10 msgr2=0x7f2b3015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 --2- 172.21.15.8:0/4047863792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b300c4a10 0x7f2b3015cf90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b1d40 tx=0x7f2b1801ec50 comp rx=0 tx=0).stop 2023-10-30T12:45:48.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 -- 172.21.15.8:0/4047863792 shutdown_connections 2023-10-30T12:45:48.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 --2- 172.21.15.8:0/4047863792 >> v2:172.21.15.46:3300/0 conn(0x7f2b300c42a0 0x7f2b300ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 --2- 172.21.15.8:0/4047863792 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b300c4a10 0x7f2b3015cf90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 -- 172.21.15.8:0/4047863792 >> 172.21.15.8:0/4047863792 conn(0x7f2b300715c0 msgr2=0x7f2b300719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 -- 172.21.15.8:0/4047863792 shutdown_connections 2023-10-30T12:45:48.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.777+0000 7f2b3593e640 1 -- 172.21.15.8:0/4047863792 wait complete. 2023-10-30T12:45:48.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 Processor -- start 2023-10-30T12:45:48.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- start start 2023-10-30T12:45:48.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b300c42a0 0x7f2b30154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 0x7f2b300cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3014fd00 con 0x7f2b30153c80 2023-10-30T12:45:48.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3014ffe0 con 0x7f2b300c4a10 2023-10-30T12:45:48.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b301502c0 con 0x7f2b300c42a0 2023-10-30T12:45:48.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53484/0 (socket says 172.21.15.8:53484) 2023-10-30T12:45:48.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 -- 172.21.15.8:0/3149352874 learned_addr learned my addr 172.21.15.8:0/3149352874 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:48.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b27fff640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 0x7f2b300cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2effd640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b300c42a0 0x7f2b30154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 -- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b300c42a0 msgr2=0x7f2b30154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b300c42a0 0x7f2b30154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 -- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 msgr2=0x7f2b300cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 0x7f2b300cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 -- 172.21.15.8:0/3149352874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009420 con 0x7f2b30153c80 2023-10-30T12:45:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b27fff640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 0x7f2b300cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:45:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2f7fe640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b28007ef0 tx=0x7f2b2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28003ba0 con 0x7f2b30153c80 2023-10-30T12:45:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2800f040 con 0x7f2b30153c80 2023-10-30T12:45:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800de80 con 0x7f2b30153c80 2023-10-30T12:45:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b300b37e0 con 0x7f2b30153c80 2023-10-30T12:45:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.781+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b300b3da0 con 0x7f2b30153c80 2023-10-30T12:45:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3004e170 con 0x7f2b30153c80 2023-10-30T12:45:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b28004030 con 0x7f2b30153c80 2023-10-30T12:45:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b2cff9640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 0x7f2b10068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b28084290 con 0x7f2b30153c80 2023-10-30T12:45:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b2effd640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 0x7f2b10068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.785+0000 7f2b2effd640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 0x7f2b10068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005890 tx=0x7f2b20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:48.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.789+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b28051690 con 0x7f2b30153c80 2023-10-30T12:45:48.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.949+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b3015dbe0 con 0x7f2b100664a0 2023-10-30T12:45:48.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b2cff9640 1 -- 172.21.15.8:0/3149352874 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264634 (secure 0 0 0) 0x7f2b20001380 con 0x7f2b100664a0 2023-10-30T12:45:48.961 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:48.963 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:45:48.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 msgr2=0x7f2b10068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 0x7f2b10068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005890 tx=0x7f2b20009040 comp rx=0 tx=0).stop 2023-10-30T12:45:48.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 msgr2=0x7f2b300b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:48.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b28007ef0 tx=0x7f2b2800d4a0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 shutdown_connections 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b100664a0 0x7f2b10068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b300c4a10 0x7f2b300cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b30153c80 0x7f2b300b42c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 --2- 172.21.15.8:0/3149352874 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b300c42a0 0x7f2b30154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:48.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 >> 172.21.15.8:0/3149352874 conn(0x7f2b300715c0 msgr2=0x7f2b30159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:48.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 shutdown_connections 2023-10-30T12:45:48.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:48.957+0000 7f2b3593e640 1 -- 172.21.15.8:0/3149352874 wait complete. 2023-10-30T12:45:48.981 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-30T12:45:47.618606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252324,"kb_used_data":2916,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722396,"statfs":{"total":773094113280,"available":772835733504,"internally_reserved":0,"allocated":2985984,"data_stored":1759192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071955+0000","last_change":"2023-10-30T12:45:19.107136+0000","last_active":"2023-10-30T12:45:20.071955+0000","last_peered":"2023-10-30T12:45:20.071955+0000","last_clean":"2023-10-30T12:45:20.071955+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:45:20.071955+0000","last_undegraded":"2023-10-30T12:45:20.071955+0000","last_fullsized":"2023-10-30T12:45:20.071955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:33:48.910689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067349+0000","last_change":"2023-10-30T12:45:19.086269+0000","last_active":"2023-10-30T12:45:20.067349+0000","last_peered":"2023-10-30T12:45:20.067349+0000","last_clean":"2023-10-30T12:45:20.067349+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:45:20.067349+0000","last_undegraded":"2023-10-30T12:45:20.067349+0000","last_fullsized":"2023-10-30T12:45:20.067349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:34:34.791535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470964+0000","last_change":"2023-10-30T12:45:19.098146+0000","last_active":"2023-10-30T12:45:20.470964+0000","last_peered":"2023-10-30T12:45:20.470964+0000","last_clean":"2023-10-30T12:45:20.470964+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:45:20.470964+0000","last_undegraded":"2023-10-30T12:45:20.470964+0000","last_fullsized":"2023-10-30T12:45:20.470964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:45:42.325441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377353+0000","last_change":"2023-10-30T12:45:19.126527+0000","last_active":"2023-10-30T12:45:20.377353+0000","last_peered":"2023-10-30T12:45:20.377353+0000","last_clean":"2023-10-30T12:45:20.377353+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:45:20.377353+0000","last_undegraded":"2023-10-30T12:45:20.377353+0000","last_fullsized":"2023-10-30T12:45:20.377353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:26:59.681149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069882+0000","last_change":"2023-10-30T12:45:19.114422+0000","last_active":"2023-10-30T12:45:20.069882+0000","last_peered":"2023-10-30T12:45:20.069882+0000","last_clean":"2023-10-30T12:45:20.069882+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:45:20.069882+0000","last_undegraded":"2023-10-30T12:45:20.069882+0000","last_fullsized":"2023-10-30T12:45:20.069882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:57:09.263782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376997+0000","last_change":"2023-10-30T12:45:19.120948+0000","last_active":"2023-10-30T12:45:20.376997+0000","last_peered":"2023-10-30T12:45:20.376997+0000","last_clean":"2023-10-30T12:45:20.376997+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:45:20.376997+0000","last_undegraded":"2023-10-30T12:45:20.376997+0000","last_fullsized":"2023-10-30T12:45:20.376997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:48:04.677113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069348+0000","last_change":"2023-10-30T12:45:19.108497+0000","last_active":"2023-10-30T12:45:20.069348+0000","last_peered":"2023-10-30T12:45:20.069348+0000","last_clean":"2023-10-30T12:45:20.069348+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:45:20.069348+0000","last_undegraded":"2023-10-30T12:45:20.069348+0000","last_fullsized":"2023-10-30T12:45:20.069348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:43:52.826848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470720+0000","last_change":"2023-10-30T12:45:19.121508+0000","last_active":"2023-10-30T12:45:20.470720+0000","last_peered":"2023-10-30T12:45:20.470720+0000","last_clean":"2023-10-30T12:45:20.470720+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:45:20.470720+0000","last_undegraded":"2023-10-30T12:45:20.470720+0000","last_fullsized":"2023-10-30T12:45:20.470720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:27:17.979749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072297+0000","last_change":"2023-10-30T12:45:19.135620+0000","last_active":"2023-10-30T12:45:20.072297+0000","last_peered":"2023-10-30T12:45:20.072297+0000","last_clean":"2023-10-30T12:45:20.072297+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:45:20.072297+0000","last_undegraded":"2023-10-30T12:45:20.072297+0000","last_fullsized":"2023-10-30T12:45:20.072297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:28.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471673+0000","last_change":"2023-10-30T12:45:19.127656+0000","last_active":"2023-10-30T12:45:20.471673+0000","last_peered":"2023-10-30T12:45:20.471673+0000","last_clean":"2023-10-30T12:45:20.471673+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:45:20.471673+0000","last_undegraded":"2023-10-30T12:45:20.471673+0000","last_fullsized":"2023-10-30T12:45:20.471673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:06:39.736671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471336+0000","last_change":"2023-10-30T12:45:19.127251+0000","last_active":"2023-10-30T12:45:20.471336+0000","last_peered":"2023-10-30T12:45:20.471336+0000","last_clean":"2023-10-30T12:45:20.471336+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:45:20.471336+0000","last_undegraded":"2023-10-30T12:45:20.471336+0000","last_fullsized":"2023-10-30T12:45:20.471336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:02:55.158597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071808+0000","last_change":"2023-10-30T12:45:19.102965+0000","last_active":"2023-10-30T12:45:20.071808+0000","last_peered":"2023-10-30T12:45:20.071808+0000","last_clean":"2023-10-30T12:45:20.071808+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:45:20.071808+0000","last_undegraded":"2023-10-30T12:45:20.071808+0000","last_fullsized":"2023-10-30T12:45:20.071808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:18:07.085147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470864+0000","last_change":"2023-10-30T12:45:19.127996+0000","last_active":"2023-10-30T12:45:20.470864+0000","last_peered":"2023-10-30T12:45:20.470864+0000","last_clean":"2023-10-30T12:45:20.470864+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:45:20.470864+0000","last_undegraded":"2023-10-30T12:45:20.470864+0000","last_fullsized":"2023-10-30T12:45:20.470864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:45:04.008631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470638+0000","last_change":"2023-10-30T12:45:19.096868+0000","last_active":"2023-10-30T12:45:20.470638+0000","last_peered":"2023-10-30T12:45:20.470638+0000","last_clean":"2023-10-30T12:45:20.470638+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:45:20.470638+0000","last_undegraded":"2023-10-30T12:45:20.470638+0000","last_fullsized":"2023-10-30T12:45:20.470638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:02:04.825098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.094477+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:15:23.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376825+0000","last_change":"2023-10-30T12:45:19.108294+0000","last_active":"2023-10-30T12:45:20.376825+0000","last_peered":"2023-10-30T12:45:20.376825+0000","last_clean":"2023-10-30T12:45:20.376825+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:45:20.376825+0000","last_undegraded":"2023-10-30T12:45:20.376825+0000","last_fullsized":"2023-10-30T12:45:20.376825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:48:16.344709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377969+0000","last_change":"2023-10-30T12:45:19.124698+0000","last_active":"2023-10-30T12:45:20.377969+0000","last_peered":"2023-10-30T12:45:20.377969+0000","last_clean":"2023-10-30T12:45:20.377969+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:45:20.377969+0000","last_undegraded":"2023-10-30T12:45:20.377969+0000","last_fullsized":"2023-10-30T12:45:20.377969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:08.215943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471127+0000","last_change":"2023-10-30T12:45:19.096395+0000","last_active":"2023-10-30T12:45:20.471127+0000","last_peered":"2023-10-30T12:45:20.471127+0000","last_clean":"2023-10-30T12:45:20.471127+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:45:20.471127+0000","last_undegraded":"2023-10-30T12:45:20.471127+0000","last_fullsized":"2023-10-30T12:45:20.471127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:08:05.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070768+0000","last_change":"2023-10-30T12:45:19.108865+0000","last_active":"2023-10-30T12:45:20.070768+0000","last_peered":"2023-10-30T12:45:20.070768+0000","last_clean":"2023-10-30T12:45:20.070768+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:45:20.070768+0000","last_undegraded":"2023-10-30T12:45:20.070768+0000","last_fullsized":"2023-10-30T12:45:20.070768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:00:17.483158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471411+0000","last_change":"2023-10-30T12:45:19.098016+0000","last_active":"2023-10-30T12:45:20.471411+0000","last_peered":"2023-10-30T12:45:20.471411+0000","last_clean":"2023-10-30T12:45:20.471411+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:45:20.471411+0000","last_undegraded":"2023-10-30T12:45:20.471411+0000","last_fullsized":"2023-10-30T12:45:20.471411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:16:21.392983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470962+0000","last_change":"2023-10-30T12:45:19.120414+0000","last_active":"2023-10-30T12:45:20.470962+0000","last_peered":"2023-10-30T12:45:20.470962+0000","last_clean":"2023-10-30T12:45:20.470962+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:45:20.470962+0000","last_undegraded":"2023-10-30T12:45:20.470962+0000","last_fullsized":"2023-10-30T12:45:20.470962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:02.564276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377142+0000","last_change":"2023-10-30T12:45:19.122406+0000","last_active":"2023-10-30T12:45:20.377142+0000","last_peered":"2023-10-30T12:45:20.377142+0000","last_clean":"2023-10-30T12:45:20.377142+0000","last_became_active":"2023-10-30T12:45:19.121698+0000","last_became_peered":"2023-10-30T12:45:19.121698+0000","last_unstale":"2023-10-30T12:45:20.377142+0000","last_undegraded":"2023-10-30T12:45:20.377142+0000","last_fullsized":"2023-10-30T12:45:20.377142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:33:03.871801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470206+0000","last_change":"2023-10-30T12:45:19.094753+0000","last_active":"2023-10-30T12:45:20.470206+0000","last_peered":"2023-10-30T12:45:20.470206+0000","last_clean":"2023-10-30T12:45:20.470206+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:45:20.470206+0000","last_undegraded":"2023-10-30T12:45:20.470206+0000","last_fullsized":"2023-10-30T12:45:20.470206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:31:49.660950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.376996+0000","last_change":"2023-10-30T12:45:19.107483+0000","last_active":"2023-10-30T12:45:20.376996+0000","last_peered":"2023-10-30T12:45:20.376996+0000","last_clean":"2023-10-30T12:45:20.376996+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:45:20.376996+0000","last_undegraded":"2023-10-30T12:45:20.376996+0000","last_fullsized":"2023-10-30T12:45:20.376996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:49:51.286544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471712+0000","last_change":"2023-10-30T12:45:19.098760+0000","last_active":"2023-10-30T12:45:20.471712+0000","last_peered":"2023-10-30T12:45:20.471712+0000","last_clean":"2023-10-30T12:45:20.471712+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:45:20.471712+0000","last_undegraded":"2023-10-30T12:45:20.471712+0000","last_fullsized":"2023-10-30T12:45:20.471712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:57.636148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471671+0000","last_change":"2023-10-30T12:45:19.111252+0000","last_active":"2023-10-30T12:45:20.471671+0000","last_peered":"2023-10-30T12:45:20.471671+0000","last_clean":"2023-10-30T12:45:20.471671+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:45:20.471671+0000","last_undegraded":"2023-10-30T12:45:20.471671+0000","last_fullsized":"2023-10-30T12:45:20.471671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:30.410457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067039+0000","last_change":"2023-10-30T12:45:19.082132+0000","last_active":"2023-10-30T12:45:20.067039+0000","last_peered":"2023-10-30T12:45:20.067039+0000","last_clean":"2023-10-30T12:45:20.067039+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:45:20.067039+0000","last_undegraded":"2023-10-30T12:45:20.067039+0000","last_fullsized":"2023-10-30T12:45:20.067039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:46.788124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471494+0000","last_change":"2023-10-30T12:45:19.094581+0000","last_active":"2023-10-30T12:45:20.471494+0000","last_peered":"2023-10-30T12:45:20.471494+0000","last_clean":"2023-10-30T12:45:20.471494+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:45:20.471494+0000","last_undegraded":"2023-10-30T12:45:20.471494+0000","last_fullsized":"2023-10-30T12:45:20.471494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:44.555138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471084+0000","last_change":"2023-10-30T12:45:19.093094+0000","last_active":"2023-10-30T12:45:20.471084+0000","last_peered":"2023-10-30T12:45:20.471084+0000","last_clean":"2023-10-30T12:45:20.471084+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:45:20.471084+0000","last_undegraded":"2023-10-30T12:45:20.471084+0000","last_fullsized":"2023-10-30T12:45:20.471084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:29.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377430+0000","last_change":"2023-10-30T12:45:19.107117+0000","last_active":"2023-10-30T12:45:20.377430+0000","last_peered":"2023-10-30T12:45:20.377430+0000","last_clean":"2023-10-30T12:45:20.377430+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:45:20.377430+0000","last_undegraded":"2023-10-30T12:45:20.377430+0000","last_fullsized":"2023-10-30T12:45:20.377430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:50:53.563609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471353+0000","last_change":"2023-10-30T12:45:19.107626+0000","last_active":"2023-10-30T12:45:20.471353+0000","last_peered":"2023-10-30T12:45:20.471353+0000","last_clean":"2023-10-30T12:45:20.471353+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:45:20.471353+0000","last_undegraded":"2023-10-30T12:45:20.471353+0000","last_fullsized":"2023-10-30T12:45:20.471353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:12:59.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470912+0000","last_change":"2023-10-30T12:45:19.088147+0000","last_active":"2023-10-30T12:45:20.470912+0000","last_peered":"2023-10-30T12:45:20.470912+0000","last_clean":"2023-10-30T12:45:20.470912+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:45:20.470912+0000","last_undegraded":"2023-10-30T12:45:20.470912+0000","last_fullsized":"2023-10-30T12:45:20.470912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:14:10.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071561+0000","last_change":"2023-10-30T12:45:19.093945+0000","last_active":"2023-10-30T12:45:20.071561+0000","last_peered":"2023-10-30T12:45:20.071561+0000","last_clean":"2023-10-30T12:45:20.071561+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:45:20.071561+0000","last_undegraded":"2023-10-30T12:45:20.071561+0000","last_fullsized":"2023-10-30T12:45:20.071561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:17:56.183441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.167894+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:45:29.167894+0000","last_peered":"2023-10-30T12:45:29.167894+0000","last_clean":"2023-10-30T12:45:29.167894+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:45:29.167894+0000","last_undegraded":"2023-10-30T12:45:29.167894+0000","last_fullsized":"2023-10-30T12:45:29.167894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471141+0000","last_change":"2023-10-30T12:45:19.121990+0000","last_active":"2023-10-30T12:45:20.471141+0000","last_peered":"2023-10-30T12:45:20.471141+0000","last_clean":"2023-10-30T12:45:20.471141+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:45:20.471141+0000","last_undegraded":"2023-10-30T12:45:20.471141+0000","last_fullsized":"2023-10-30T12:45:20.471141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:23:00.831740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203892+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:45:35.203892+0000","last_peered":"2023-10-30T12:45:35.203892+0000","last_clean":"2023-10-30T12:45:35.203892+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:45:35.203892+0000","last_undegraded":"2023-10-30T12:45:35.203892+0000","last_fullsized":"2023-10-30T12:45:35.203892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.203844+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:45:35.203844+0000","last_peered":"2023-10-30T12:45:35.203844+0000","last_clean":"2023-10-30T12:45:35.203844+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:45:35.203844+0000","last_undegraded":"2023-10-30T12:45:35.203844+0000","last_fullsized":"2023-10-30T12:45:35.203844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470208+0000","last_change":"2023-10-30T12:45:19.088790+0000","last_active":"2023-10-30T12:45:20.470208+0000","last_peered":"2023-10-30T12:45:20.470208+0000","last_clean":"2023-10-30T12:45:20.470208+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:45:20.470208+0000","last_undegraded":"2023-10-30T12:45:20.470208+0000","last_fullsized":"2023-10-30T12:45:20.470208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:26.174484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377740+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:45:20.377740+0000","last_peered":"2023-10-30T12:45:20.377740+0000","last_clean":"2023-10-30T12:45:20.377740+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:45:20.377740+0000","last_undegraded":"2023-10-30T12:45:20.377740+0000","last_fullsized":"2023-10-30T12:45:20.377740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066844+0000","last_change":"2023-10-30T12:45:19.079709+0000","last_active":"2023-10-30T12:45:20.066844+0000","last_peered":"2023-10-30T12:45:20.066844+0000","last_clean":"2023-10-30T12:45:20.066844+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:45:20.066844+0000","last_undegraded":"2023-10-30T12:45:20.066844+0000","last_fullsized":"2023-10-30T12:45:20.066844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:12:12.444137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157838+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:45:29.157838+0000","last_peered":"2023-10-30T12:45:29.157838+0000","last_clean":"2023-10-30T12:45:29.157838+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:45:29.157838+0000","last_undegraded":"2023-10-30T12:45:29.157838+0000","last_fullsized":"2023-10-30T12:45:29.157838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377703+0000","last_change":"2023-10-30T12:45:19.122874+0000","last_active":"2023-10-30T12:45:20.377703+0000","last_peered":"2023-10-30T12:45:20.377703+0000","last_clean":"2023-10-30T12:45:20.377703+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:45:20.377703+0000","last_undegraded":"2023-10-30T12:45:20.377703+0000","last_fullsized":"2023-10-30T12:45:20.377703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:40:41.849174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.158692+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:45:29.158692+0000","last_peered":"2023-10-30T12:45:29.158692+0000","last_clean":"2023-10-30T12:45:29.158692+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:45:29.158692+0000","last_undegraded":"2023-10-30T12:45:29.158692+0000","last_fullsized":"2023-10-30T12:45:29.158692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471307+0000","last_change":"2023-10-30T12:45:19.131072+0000","last_active":"2023-10-30T12:45:20.471307+0000","last_peered":"2023-10-30T12:45:20.471307+0000","last_clean":"2023-10-30T12:45:20.471307+0000","last_became_active":"2023-10-30T12:45:19.130862+0000","last_became_peered":"2023-10-30T12:45:19.130862+0000","last_unstale":"2023-10-30T12:45:20.471307+0000","last_undegraded":"2023-10-30T12:45:20.471307+0000","last_fullsized":"2023-10-30T12:45:20.471307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:07:09.393386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.086171+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:45:24.086171+0000","last_peered":"2023-10-30T12:45:24.086171+0000","last_clean":"2023-10-30T12:45:24.086171+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:45:24.086171+0000","last_undegraded":"2023-10-30T12:45:24.086171+0000","last_fullsized":"2023-10-30T12:45:24.086171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072148+0000","last_change":"2023-10-30T12:45:19.116645+0000","last_active":"2023-10-30T12:45:20.072148+0000","last_peered":"2023-10-30T12:45:20.072148+0000","last_clean":"2023-10-30T12:45:20.072148+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:45:20.072148+0000","last_undegraded":"2023-10-30T12:45:20.072148+0000","last_fullsized":"2023-10-30T12:45:20.072148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:17.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'11","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:43.496620+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:45:43.496620+0000","last_peered":"2023-10-30T12:45:43.496620+0000","last_clean":"2023-10-30T12:45:43.496620+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:45:43.496620+0000","last_undegraded":"2023-10-30T12:45:43.496620+0000","last_fullsized":"2023-10-30T12:45:43.496620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069876+0000","last_change":"2023-10-30T12:45:19.113338+0000","last_active":"2023-10-30T12:45:20.069876+0000","last_peered":"2023-10-30T12:45:20.069876+0000","last_clean":"2023-10-30T12:45:20.069876+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:45:20.069876+0000","last_undegraded":"2023-10-30T12:45:20.069876+0000","last_fullsized":"2023-10-30T12:45:20.069876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:07:42.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201182+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:45:35.201182+0000","last_peered":"2023-10-30T12:45:35.201182+0000","last_clean":"2023-10-30T12:45:35.201182+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:45:35.201182+0000","last_undegraded":"2023-10-30T12:45:35.201182+0000","last_fullsized":"2023-10-30T12:45:35.201182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470757+0000","last_change":"2023-10-30T12:45:19.120848+0000","last_active":"2023-10-30T12:45:20.470757+0000","last_peered":"2023-10-30T12:45:20.470757+0000","last_clean":"2023-10-30T12:45:20.470757+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:45:20.470757+0000","last_undegraded":"2023-10-30T12:45:20.470757+0000","last_fullsized":"2023-10-30T12:45:20.470757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:16:17.831426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.164676+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:45:29.164676+0000","last_peered":"2023-10-30T12:45:29.164676+0000","last_clean":"2023-10-30T12:45:29.164676+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:45:29.164676+0000","last_undegraded":"2023-10-30T12:45:29.164676+0000","last_fullsized":"2023-10-30T12:45:29.164676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.070431+0000","last_change":"2023-10-30T12:45:19.124511+0000","last_active":"2023-10-30T12:45:20.070431+0000","last_peered":"2023-10-30T12:45:20.070431+0000","last_clean":"2023-10-30T12:45:20.070431+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:45:20.070431+0000","last_undegraded":"2023-10-30T12:45:20.070431+0000","last_fullsized":"2023-10-30T12:45:20.070431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:14:31.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.210377+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:45:35.210377+0000","last_peered":"2023-10-30T12:45:35.210377+0000","last_clean":"2023-10-30T12:45:35.210377+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:45:35.210377+0000","last_undegraded":"2023-10-30T12:45:35.210377+0000","last_fullsized":"2023-10-30T12:45:35.210377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470957+0000","last_change":"2023-10-30T12:45:19.087097+0000","last_active":"2023-10-30T12:45:20.470957+0000","last_peered":"2023-10-30T12:45:20.470957+0000","last_clean":"2023-10-30T12:45:20.470957+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:45:20.470957+0000","last_undegraded":"2023-10-30T12:45:20.470957+0000","last_fullsized":"2023-10-30T12:45:20.470957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:16:13.282854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'15","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201839+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:45:35.201839+0000","last_peered":"2023-10-30T12:45:35.201839+0000","last_clean":"2023-10-30T12:45:35.201839+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:45:35.201839+0000","last_undegraded":"2023-10-30T12:45:35.201839+0000","last_fullsized":"2023-10-30T12:45:35.201839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069448+0000","last_change":"2023-10-30T12:45:19.108442+0000","last_active":"2023-10-30T12:45:20.069448+0000","last_peered":"2023-10-30T12:45:20.069448+0000","last_clean":"2023-10-30T12:45:20.069448+0000","last_became_active":"2023-10-30T12:45:19.107553+0000","last_became_peered":"2023-10-30T12:45:19.107553+0000","last_unstale":"2023-10-30T12:45:20.069448+0000","last_undegraded":"2023-10-30T12:45:20.069448+0000","last_fullsized":"2023-10-30T12:45:20.069448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:37:14.563685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.079122+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:45:24.079122+0000","last_peered":"2023-10-30T12:45:24.079122+0000","last_clean":"2023-10-30T12:45:24.079122+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:45:24.079122+0000","last_undegraded":"2023-10-30T12:45:24.079122+0000","last_fullsized":"2023-10-30T12:45:24.079122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071276+0000","last_change":"2023-10-30T12:45:19.107093+0000","last_active":"2023-10-30T12:45:20.071276+0000","last_peered":"2023-10-30T12:45:20.071276+0000","last_clean":"2023-10-30T12:45:20.071276+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:45:20.071276+0000","last_undegraded":"2023-10-30T12:45:20.071276+0000","last_fullsized":"2023-10-30T12:45:20.071276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:31:43.133050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.077908+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:45:24.077908+0000","last_peered":"2023-10-30T12:45:24.077908+0000","last_clean":"2023-10-30T12:45:24.077908+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:45:24.077908+0000","last_undegraded":"2023-10-30T12:45:24.077908+0000","last_fullsized":"2023-10-30T12:45:24.077908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072543+0000","last_change":"2023-10-30T12:45:19.108773+0000","last_active":"2023-10-30T12:45:20.072543+0000","last_peered":"2023-10-30T12:45:20.072543+0000","last_clean":"2023-10-30T12:45:20.072543+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:45:20.072543+0000","last_undegraded":"2023-10-30T12:45:20.072543+0000","last_fullsized":"2023-10-30T12:45:20.072543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:54.641020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067271+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:45:20.067271+0000","last_peered":"2023-10-30T12:45:20.067271+0000","last_clean":"2023-10-30T12:45:20.067271+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:45:20.067271+0000","last_undegraded":"2023-10-30T12:45:20.067271+0000","last_fullsized":"2023-10-30T12:45:20.067271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378262+0000","last_change":"2023-10-30T12:45:19.124215+0000","last_active":"2023-10-30T12:45:20.378262+0000","last_peered":"2023-10-30T12:45:20.378262+0000","last_clean":"2023-10-30T12:45:20.378262+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:45:20.378262+0000","last_undegraded":"2023-10-30T12:45:20.378262+0000","last_fullsized":"2023-10-30T12:45:20.378262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:05:08.395587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.205713+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:45:35.205713+0000","last_peered":"2023-10-30T12:45:35.205713+0000","last_clean":"2023-10-30T12:45:35.205713+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:45:35.205713+0000","last_undegraded":"2023-10-30T12:45:35.205713+0000","last_fullsized":"2023-10-30T12:45:35.205713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.069897+0000","last_change":"2023-10-30T12:45:19.126919+0000","last_active":"2023-10-30T12:45:20.069897+0000","last_peered":"2023-10-30T12:45:20.069897+0000","last_clean":"2023-10-30T12:45:20.069897+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:45:20.069897+0000","last_undegraded":"2023-10-30T12:45:20.069897+0000","last_fullsized":"2023-10-30T12:45:20.069897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:58:00.569432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.071157+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:45:19.071157+0000","last_peered":"2023-10-30T12:45:19.071157+0000","last_clean":"2023-10-30T12:45:19.071157+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:45:19.071157+0000","last_undegraded":"2023-10-30T12:45:19.071157+0000","last_fullsized":"2023-10-30T12:45:19.071157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156996+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:45:29.156996+0000","last_peered":"2023-10-30T12:45:29.156996+0000","last_clean":"2023-10-30T12:45:29.156996+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:45:29.156996+0000","last_undegraded":"2023-10-30T12:45:29.156996+0000","last_fullsized":"2023-10-30T12:45:29.156996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470630+0000","last_change":"2023-10-30T12:45:19.124678+0000","last_active":"2023-10-30T12:45:20.470630+0000","last_peered":"2023-10-30T12:45:20.470630+0000","last_clean":"2023-10-30T12:45:20.470630+0000","last_became_active":"2023-10-30T12:45:19.124338+0000","last_became_peered":"2023-10-30T12:45:19.124338+0000","last_unstale":"2023-10-30T12:45:20.470630+0000","last_undegraded":"2023-10-30T12:45:20.470630+0000","last_fullsized":"2023-10-30T12:45:20.470630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:25:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201698+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:45:35.201698+0000","last_peered":"2023-10-30T12:45:35.201698+0000","last_clean":"2023-10-30T12:45:35.201698+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:45:35.201698+0000","last_undegraded":"2023-10-30T12:45:35.201698+0000","last_fullsized":"2023-10-30T12:45:35.201698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470604+0000","last_change":"2023-10-30T12:45:19.121571+0000","last_active":"2023-10-30T12:45:20.470604+0000","last_peered":"2023-10-30T12:45:20.470604+0000","last_clean":"2023-10-30T12:45:20.470604+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:45:20.470604+0000","last_undegraded":"2023-10-30T12:45:20.470604+0000","last_fullsized":"2023-10-30T12:45:20.470604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:42:55.822788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.202222+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:45:35.202222+0000","last_peered":"2023-10-30T12:45:35.202222+0000","last_clean":"2023-10-30T12:45:35.202222+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:45:35.202222+0000","last_undegraded":"2023-10-30T12:45:35.202222+0000","last_fullsized":"2023-10-30T12:45:35.202222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066971+0000","last_change":"2023-10-30T12:45:19.075600+0000","last_active":"2023-10-30T12:45:20.066971+0000","last_peered":"2023-10-30T12:45:20.066971+0000","last_clean":"2023-10-30T12:45:20.066971+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:45:20.066971+0000","last_undegraded":"2023-10-30T12:45:20.066971+0000","last_fullsized":"2023-10-30T12:45:20.066971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:17:16.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377841+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:45:20.377841+0000","last_peered":"2023-10-30T12:45:20.377841+0000","last_clean":"2023-10-30T12:45:20.377841+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:45:20.377841+0000","last_undegraded":"2023-10-30T12:45:20.377841+0000","last_fullsized":"2023-10-30T12:45:20.377841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.072149+0000","last_change":"2023-10-30T12:45:19.119421+0000","last_active":"2023-10-30T12:45:20.072149+0000","last_peered":"2023-10-30T12:45:20.072149+0000","last_clean":"2023-10-30T12:45:20.072149+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:45:20.072149+0000","last_undegraded":"2023-10-30T12:45:20.072149+0000","last_fullsized":"2023-10-30T12:45:20.072149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:18:31.953464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-30T12:45:19.630601+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:45:19.630601+0000","last_peered":"2023-10-30T12:45:19.630601+0000","last_clean":"2023-10-30T12:45:19.630601+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:45:19.630601+0000","last_undegraded":"2023-10-30T12:45:19.630601+0000","last_fullsized":"2023-10-30T12:45:19.630601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.378093+0000","last_change":"2023-10-30T12:45:19.120601+0000","last_active":"2023-10-30T12:45:20.378093+0000","last_peered":"2023-10-30T12:45:20.378093+0000","last_clean":"2023-10-30T12:45:20.378093+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:45:20.378093+0000","last_undegraded":"2023-10-30T12:45:20.378093+0000","last_fullsized":"2023-10-30T12:45:20.378093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:05.357577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071109+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:45:20.071109+0000","last_peered":"2023-10-30T12:45:20.071109+0000","last_clean":"2023-10-30T12:45:20.071109+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:45:20.071109+0000","last_undegraded":"2023-10-30T12:45:20.071109+0000","last_fullsized":"2023-10-30T12:45:20.071109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.066974+0000","last_change":"2023-10-30T12:45:19.079477+0000","last_active":"2023-10-30T12:45:20.066974+0000","last_peered":"2023-10-30T12:45:20.066974+0000","last_clean":"2023-10-30T12:45:20.066974+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:45:20.066974+0000","last_undegraded":"2023-10-30T12:45:20.066974+0000","last_fullsized":"2023-10-30T12:45:20.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:46:43.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.201094+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:45:35.201094+0000","last_peered":"2023-10-30T12:45:35.201094+0000","last_clean":"2023-10-30T12:45:35.201094+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:45:35.201094+0000","last_undegraded":"2023-10-30T12:45:35.201094+0000","last_fullsized":"2023-10-30T12:45:35.201094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471700+0000","last_change":"2023-10-30T12:45:19.112651+0000","last_active":"2023-10-30T12:45:20.471700+0000","last_peered":"2023-10-30T12:45:20.471700+0000","last_clean":"2023-10-30T12:45:20.471700+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:45:20.471700+0000","last_undegraded":"2023-10-30T12:45:20.471700+0000","last_fullsized":"2023-10-30T12:45:20.471700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:42:25.320280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.157353+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:45:29.157353+0000","last_peered":"2023-10-30T12:45:29.157353+0000","last_clean":"2023-10-30T12:45:29.157353+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:45:29.157353+0000","last_undegraded":"2023-10-30T12:45:29.157353+0000","last_fullsized":"2023-10-30T12:45:29.157353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471488+0000","last_change":"2023-10-30T12:45:19.091008+0000","last_active":"2023-10-30T12:45:20.471488+0000","last_peered":"2023-10-30T12:45:20.471488+0000","last_clean":"2023-10-30T12:45:20.471488+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:45:20.471488+0000","last_undegraded":"2023-10-30T12:45:20.471488+0000","last_fullsized":"2023-10-30T12:45:20.471488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:15:14.407746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.156619+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:45:29.156619+0000","last_peered":"2023-10-30T12:45:29.156619+0000","last_clean":"2023-10-30T12:45:29.156619+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:45:29.156619+0000","last_undegraded":"2023-10-30T12:45:29.156619+0000","last_fullsized":"2023-10-30T12:45:29.156619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.071773+0000","last_change":"2023-10-30T12:45:19.092651+0000","last_active":"2023-10-30T12:45:20.071773+0000","last_peered":"2023-10-30T12:45:20.071773+0000","last_clean":"2023-10-30T12:45:20.071773+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:45:20.071773+0000","last_undegraded":"2023-10-30T12:45:20.071773+0000","last_fullsized":"2023-10-30T12:45:20.071773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:09.037055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.165872+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:45:29.165872+0000","last_peered":"2023-10-30T12:45:29.165872+0000","last_clean":"2023-10-30T12:45:29.165872+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:45:29.165872+0000","last_undegraded":"2023-10-30T12:45:29.165872+0000","last_fullsized":"2023-10-30T12:45:29.165872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470363+0000","last_change":"2023-10-30T12:45:19.077338+0000","last_active":"2023-10-30T12:45:20.470363+0000","last_peered":"2023-10-30T12:45:20.470363+0000","last_clean":"2023-10-30T12:45:20.470363+0000","last_became_active":"2023-10-30T12:45:19.076920+0000","last_became_peered":"2023-10-30T12:45:19.076920+0000","last_unstale":"2023-10-30T12:45:20.470363+0000","last_undegraded":"2023-10-30T12:45:20.470363+0000","last_fullsized":"2023-10-30T12:45:20.470363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:45:36.069824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.067111+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:45:20.067111+0000","last_peered":"2023-10-30T12:45:20.067111+0000","last_clean":"2023-10-30T12:45:20.067111+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:45:20.067111+0000","last_undegraded":"2023-10-30T12:45:20.067111+0000","last_fullsized":"2023-10-30T12:45:20.067111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470988+0000","last_change":"2023-10-30T12:45:19.098331+0000","last_active":"2023-10-30T12:45:20.470988+0000","last_peered":"2023-10-30T12:45:20.470988+0000","last_clean":"2023-10-30T12:45:20.470988+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:45:20.470988+0000","last_undegraded":"2023-10-30T12:45:20.470988+0000","last_fullsized":"2023-10-30T12:45:20.470988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:21.343385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:29.151892+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:45:29.151892+0000","last_peered":"2023-10-30T12:45:29.151892+0000","last_clean":"2023-10-30T12:45:29.151892+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:45:29.151892+0000","last_undegraded":"2023-10-30T12:45:29.151892+0000","last_fullsized":"2023-10-30T12:45:29.151892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.470979+0000","last_change":"2023-10-30T12:45:19.094994+0000","last_active":"2023-10-30T12:45:20.470979+0000","last_peered":"2023-10-30T12:45:20.470979+0000","last_clean":"2023-10-30T12:45:20.470979+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:45:20.470979+0000","last_undegraded":"2023-10-30T12:45:20.470979+0000","last_fullsized":"2023-10-30T12:45:20.470979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:44.769458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377742+0000","last_change":"2023-10-30T12:45:19.131567+0000","last_active":"2023-10-30T12:45:20.377742+0000","last_peered":"2023-10-30T12:45:20.377742+0000","last_clean":"2023-10-30T12:45:20.377742+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:45:20.377742+0000","last_undegraded":"2023-10-30T12:45:20.377742+0000","last_fullsized":"2023-10-30T12:45:20.377742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:13.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.084274+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:45:24.084274+0000","last_peered":"2023-10-30T12:45:24.084274+0000","last_clean":"2023-10-30T12:45:24.084274+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:45:24.084274+0000","last_undegraded":"2023-10-30T12:45:24.084274+0000","last_fullsized":"2023-10-30T12:45:24.084274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.377958+0000","last_change":"2023-10-30T12:45:19.131665+0000","last_active":"2023-10-30T12:45:20.377958+0000","last_peered":"2023-10-30T12:45:20.377958+0000","last_clean":"2023-10-30T12:45:20.377958+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:45:20.377958+0000","last_undegraded":"2023-10-30T12:45:20.377958+0000","last_fullsized":"2023-10-30T12:45:20.377958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:08.137679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.207109+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:45:35.207109+0000","last_peered":"2023-10-30T12:45:35.207109+0000","last_clean":"2023-10-30T12:45:35.207109+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:45:35.207109+0000","last_undegraded":"2023-10-30T12:45:35.207109+0000","last_fullsized":"2023-10-30T12:45:35.207109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471292+0000","last_change":"2023-10-30T12:45:19.131245+0000","last_active":"2023-10-30T12:45:20.471292+0000","last_peered":"2023-10-30T12:45:20.471292+0000","last_clean":"2023-10-30T12:45:20.471292+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:45:20.471292+0000","last_undegraded":"2023-10-30T12:45:20.471292+0000","last_fullsized":"2023-10-30T12:45:20.471292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:30:26.117300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:24.078665+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:45:24.078665+0000","last_peered":"2023-10-30T12:45:24.078665+0000","last_clean":"2023-10-30T12:45:24.078665+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:45:24.078665+0000","last_undegraded":"2023-10-30T12:45:24.078665+0000","last_fullsized":"2023-10-30T12:45:24.078665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:20.471526+0000","last_change":"2023-10-30T12:45:19.127896+0000","last_active":"2023-10-30T12:45:20.471526+0000","last_peered":"2023-10-30T12:45:20.471526+0000","last_clean":"2023-10-30T12:45:20.471526+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:45:20.471526+0000","last_undegraded":"2023-10-30T12:45:20.471526+0000","last_fullsized":"2023-10-30T12:45:20.471526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:57:18.999410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'9","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-30T12:45:35.208649+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:45:35.208649+0000","last_peered":"2023-10-30T12:45:35.208649+0000","last_clean":"2023-10-30T12:45:35.208649+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:45:35.208649+0000","last_undegraded":"2023-10-30T12:45:35.208649+0000","last_fullsized":"2023-10-30T12:45:35.208649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":819200,"data_stored":662582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":73704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":72276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":221184,"data_stored":71870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":74295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":71716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":71842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":660907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:45:48.987 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:49.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 Processor -- start 2023-10-30T12:45:49.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 -- start start 2023-10-30T12:45:49.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bd815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bd80c42a0 0x7f3bd80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd8159f20 con 0x7f3bd80c4a10 2023-10-30T12:45:49.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd815dec0 con 0x7f3bd80c42a0 2023-10-30T12:45:49.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bdc91d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81599a0 con 0x7f3bd8153c80 2023-10-30T12:45:49.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bcffff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3bd80c4a10 0x7f3bd815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39952/0 (socket says 172.21.15.8:39952) 2023-10-30T12:45:49.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bcffff640 1 -- 172.21.15.8:0/708081159 learned_addr learned my addr 172.21.15.8:0/708081159 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:49.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd659c640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bd815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/708081159 >> v2:172.21.15.46:3300/0 conn(0x7f3bd80c42a0 0x7f3bd80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753311689 0 0) 0x7f3bd8159f20 con 0x7f3bd80c4a10 2023-10-30T12:45:49.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc4003530 con 0x7f3bd80c4a10 2023-10-30T12:45:49.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1594214181 0 0) 0x7f3bc4003530 con 0x7f3bd80c4a10 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 msgr2=0x7f3bd815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bd815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 >> v2:172.21.15.46:3300/0 conn(0x7f3bd80c42a0 msgr2=0x7f3bd80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 --2- 172.21.15.8:0/708081159 >> v2:172.21.15.46:3300/0 conn(0x7f3bd80c42a0 0x7f3bd80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd815d620 con 0x7f3bd80c4a10 2023-10-30T12:45:49.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f3bc0001680 con 0x7f3bd80c4a10 2023-10-30T12:45:49.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 >> v1:172.21.15.8:6789/0 conn(0x7f3bd80c4a10 legacy=0x7f3bd815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bc4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc4003f00 0x7f3bc40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bc4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815dec0 con 0x7f3bc4004d10 2023-10-30T12:45:49.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81599a0 con 0x7f3bc4003f00 2023-10-30T12:45:49.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc400b9c0 con 0x7f3bd8153c80 2023-10-30T12:45:49.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bc4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bd659c640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bc4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.161+0000 7f3bcffff640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc4003f00 0x7f3bc40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 -- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 msgr2=0x7f3bc4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd8153c80 0x7f3bc4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 -- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc4003f00 msgr2=0x7f3bc40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc4003f00 0x7f3bc40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 -- 172.21.15.8:0/708081159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4003530 con 0x7f3bc4004d10 2023-10-30T12:45:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bc4005190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009fc0 tx=0x7f3bd001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d18a33064a359cb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00a5cb0 con 0x7f3bc4004d10 2023-10-30T12:45:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd000f040 con 0x7f3bc4004d10 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd5d9b640 1 -- 172.21.15.8:0/708081159 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00ad680 con 0x7f3bc4004d10 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bdc91d640 1 -- 172.21.15.8:0/708081159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 msgr2=0x7f3bc4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/708081159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bc4005190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009fc0 tx=0x7f3bd001cca0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bd6d9d640 1 -- 172.21.15.8:0/708081159 reap_dead start 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bdc91d640 1 -- 172.21.15.8:0/708081159 shutdown_connections 2023-10-30T12:45:49.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.165+0000 7f3bdc91d640 1 -- 172.21.15.8:0/708081159 >> 172.21.15.8:0/708081159 conn(0x7f3bd80715c0 msgr2=0x7f3bd80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- 172.21.15.8:0/708081159 shutdown_connections 2023-10-30T12:45:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- 172.21.15.8:0/708081159 wait complete. 2023-10-30T12:45:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 Processor -- start 2023-10-30T12:45:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- start start 2023-10-30T12:45:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd80c5b10 0x7f3bd81541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd80c50c0 0x7f3bd81ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8155790 con 0x7f3bc4004d10 2023-10-30T12:45:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81546f0 con 0x7f3bd80c5b10 2023-10-30T12:45:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bdc91d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81f2440 con 0x7f3bd80c50c0 2023-10-30T12:45:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50298/0 (socket says 172.21.15.8:50298) 2023-10-30T12:45:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 -- 172.21.15.8:0/38789702 learned_addr learned my addr 172.21.15.8:0/38789702 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd6d9d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd80c50c0 0x7f3bd81ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bcffff640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd80c5b10 0x7f3bd81541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 -- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd80c50c0 msgr2=0x7f3bd81ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd80c50c0 0x7f3bd81ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 -- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd80c5b10 msgr2=0x7f3bd81541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd80c5b10 0x7f3bd81541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 -- 172.21.15.8:0/38789702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8154910 con 0x7f3bc4004d10 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.169+0000 7f3bd659c640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8011a30 tx=0x7f3bc8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800fb50 con 0x7f3bc4004d10 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800fcb0 con 0x7f3bc4004d10 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8014440 con 0x7f3bc4004d10 2023-10-30T12:45:49.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd8154bf0 con 0x7f3bc4004d10 2023-10-30T12:45:49.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd81f20d0 con 0x7f3bc4004d10 2023-10-30T12:45:49.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd804e170 con 0x7f3bc4004d10 2023-10-30T12:45:49.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3bc80145a0 con 0x7f3bc4004d10 2023-10-30T12:45:49.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 0x7f3bb8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc80847b0 con 0x7f3bc4004d10 2023-10-30T12:45:49.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bcffff640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 0x7f3bb8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.173+0000 7f3bcffff640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 0x7f3bb8068990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bd814dcc0 tx=0x7f3bc0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.177+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3bc8051c50 con 0x7f3bc4004d10 2023-10-30T12:45:49.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.337+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bd80cad40 con 0x7f3bc4004d10 2023-10-30T12:45:49.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bceffd640 1 -- 172.21.15.8:0/38789702 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f3bc8022090 con 0x7f3bc4004d10 2023-10-30T12:45:49.343 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:49.343 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 msgr2=0x7f3bb8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 0x7f3bb8068990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bd814dcc0 tx=0x7f3bc0009040 comp rx=0 tx=0).stop 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 msgr2=0x7f3bd8156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8011a30 tx=0x7f3bc8011ef0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 shutdown_connections 2023-10-30T12:45:49.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3bb80664d0 0x7f3bb8068990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bd80c5b10 0x7f3bd81541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3bd80c50c0 0x7f3bd81ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 --2- 172.21.15.8:0/38789702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3bc4004d10 0x7f3bd8156d60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 >> 172.21.15.8:0/38789702 conn(0x7f3bd80715c0 msgr2=0x7f3bd8148c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:49.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 shutdown_connections 2023-10-30T12:45:49.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.341+0000 7f3bdc91d640 1 -- 172.21.15.8:0/38789702 wait complete. 2023-10-30T12:45:49.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:45:49.365 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:45:49.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 Processor -- start 2023-10-30T12:45:49.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 -- start start 2023-10-30T12:45:49.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f30c40c1430 0x7f30c4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30c415a040 0x7f30c40cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c4157100 con 0x7f30c4150630 2023-10-30T12:45:49.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c4150a10 con 0x7f30c40c1430 2023-10-30T12:45:49.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c8d94640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c4072000 con 0x7f30c415a040 2023-10-30T12:45:49.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f30c40c1430 0x7f30c4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f30c40c1430 0x7f30c4159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50310/0 (socket says 172.21.15.8:50310) 2023-10-30T12:45:49.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 -- 172.21.15.8:0/1775578390 learned_addr learned my addr 172.21.15.8:0/1775578390 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:49.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c2d9d640 1 --2- 172.21.15.8:0/1775578390 >> v2:172.21.15.46:3300/0 conn(0x7f30c415a040 0x7f30c40cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 -- 172.21.15.8:0/1775578390 >> v2:172.21.15.46:3300/0 conn(0x7f30c415a040 msgr2=0x7f30c40cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 --2- 172.21.15.8:0/1775578390 >> v2:172.21.15.46:3300/0 conn(0x7f30c415a040 0x7f30c40cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 -- 172.21.15.8:0/1775578390 >> v1:172.21.15.46:6789/0 conn(0x7f30c4150630 legacy=0x7f30c40c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.509+0000 7f30c1d9b640 1 -- 172.21.15.8:0/1775578390 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4156510 con 0x7f30c40c1430 2023-10-30T12:45:49.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c1d9b640 1 --2- 172.21.15.8:0/1775578390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c40c1430 0x7f30c4159b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b1140 tx=0x7f30b801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c2c6f5be1dd199 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c159a640 1 -- 172.21.15.8:0/1775578390 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80ab600 con 0x7f30c40c1430 2023-10-30T12:45:49.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c159a640 1 -- 172.21.15.8:0/1775578390 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b80a7050 con 0x7f30c40c1430 2023-10-30T12:45:49.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c159a640 1 -- 172.21.15.8:0/1775578390 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80ab980 con 0x7f30c40c1430 2023-10-30T12:45:49.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 -- 172.21.15.8:0/1775578390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c40c1430 msgr2=0x7f30c4159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 --2- 172.21.15.8:0/1775578390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c40c1430 0x7f30c4159b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b1140 tx=0x7f30b801c4c0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 -- 172.21.15.8:0/1775578390 shutdown_connections 2023-10-30T12:45:49.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 --2- 172.21.15.8:0/1775578390 >> v2:172.21.15.46:3300/0 conn(0x7f30c415a040 0x7f30c40cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 --2- 172.21.15.8:0/1775578390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c40c1430 0x7f30c4159b00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 -- 172.21.15.8:0/1775578390 >> 172.21.15.8:0/1775578390 conn(0x7f30c406da40 msgr2=0x7f30c406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:49.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 -- 172.21.15.8:0/1775578390 shutdown_connections 2023-10-30T12:45:49.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.513+0000 7f30c8d94640 1 -- 172.21.15.8:0/1775578390 wait complete. 2023-10-30T12:45:49.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 Processor -- start 2023-10-30T12:45:49.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- start start 2023-10-30T12:45:49.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f30c40c1430 0x7f30c415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4150630 0x7f30c413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415fbd0 con 0x7f30c415a040 2023-10-30T12:45:49.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40c70b0 con 0x7f30c4150630 2023-10-30T12:45:49.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415f780 con 0x7f30c40c1430 2023-10-30T12:45:49.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50312/0 (socket says 172.21.15.8:50312) 2023-10-30T12:45:49.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 -- 172.21.15.8:0/3608764137 learned_addr learned my addr 172.21.15.8:0/3608764137 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:49.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c1d9b640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4150630 0x7f30c413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c259c640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f30c40c1430 0x7f30c415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 -- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f30c40c1430 msgr2=0x7f30c415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f30c40c1430 0x7f30c415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 -- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4150630 msgr2=0x7f30c413f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4150630 0x7f30c413f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 -- 172.21.15.8:0/3608764137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80094d0 con 0x7f30c415a040 2023-10-30T12:45:49.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c2d9d640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30bc007e40 tx=0x7f30bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc014620 con 0x7f30c415a040 2023-10-30T12:45:49.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc00ee90 con 0x7f30c415a040 2023-10-30T12:45:49.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0149e0 con 0x7f30c415a040 2023-10-30T12:45:49.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c41f59f0 con 0x7f30c415a040 2023-10-30T12:45:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c41f6230 con 0x7f30c415a040 2023-10-30T12:45:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.517+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c404e170 con 0x7f30c415a040 2023-10-30T12:45:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.521+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f30bc014b40 con 0x7f30c415a040 2023-10-30T12:45:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.521+0000 7f30b2ffd640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 0x7f30a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.521+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30bc084810 con 0x7f30c415a040 2023-10-30T12:45:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.521+0000 7f30c259c640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 0x7f30a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.521+0000 7f30c259c640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 0x7f30a4068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f30b4004e30 tx=0x7f30b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.525+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f30bc051bf0 con 0x7f30c415a040 2023-10-30T12:45:49.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30c40c8690 con 0x7f30c415a040 2023-10-30T12:45:49.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30b2ffd640 1 -- 172.21.15.8:0/3608764137 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30bc055210 con 0x7f30c415a040 2023-10-30T12:45:49.688 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:45:49.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 msgr2=0x7f30a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 0x7f30a4068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f30b4004e30 tx=0x7f30b4009040 comp rx=0 tx=0).stop 2023-10-30T12:45:49.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 msgr2=0x7f30c40c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30bc007e40 tx=0x7f30bc00e4b0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 shutdown_connections 2023-10-30T12:45:49.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f30a40664a0 0x7f30a4068960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4150630 0x7f30c413f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30c415a040 0x7f30c40c7620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 --2- 172.21.15.8:0/3608764137 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f30c40c1430 0x7f30c415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 >> 172.21.15.8:0/3608764137 conn(0x7f30c406da40 msgr2=0x7f30c4154380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 shutdown_connections 2023-10-30T12:45:49.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.685+0000 7f30c8d94640 1 -- 172.21.15.8:0/3608764137 wait complete. 2023-10-30T12:45:49.709 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:45:49.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.849+0000 7f3b589c5640 1 Processor -- start 2023-10-30T12:45:49.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.849+0000 7f3b589c5640 1 -- start start 2023-10-30T12:45:49.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b589c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b54153ba0 0x7f3b540cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b589c5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3b540c4410 0x7f3b5415a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b589c5640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b54153f80 con 0x7f3b540c4810 2023-10-30T12:45:49.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b589c5640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b54159df0 con 0x7f3b540c4410 2023-10-30T12:45:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b589c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5415a7c0 con 0x7f3b54153ba0 2023-10-30T12:45:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3b540c4410 0x7f3b5415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3b540c4410 0x7f3b5415a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50318/0 (socket says 172.21.15.8:50318) 2023-10-30T12:45:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 -- 172.21.15.8:0/1537615917 learned_addr learned my addr 172.21.15.8:0/1537615917 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b5259c640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b54153ba0 0x7f3b540cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 -- 172.21.15.8:0/1537615917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b54153ba0 msgr2=0x7f3b540cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b54153ba0 0x7f3b540cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 -- 172.21.15.8:0/1537615917 >> v1:172.21.15.8:6789/0 conn(0x7f3b540c4810 legacy=0x7f3b5415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b51d9b640 1 -- 172.21.15.8:0/1537615917 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b541597c0 con 0x7f3b540c4410 2023-10-30T12:45:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.853+0000 7f3b5159a640 1 -- 172.21.15.8:0/1537615917 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487222665 0 0) 0x7f3b54153f80 con 0x7f3b540c4810 2023-10-30T12:45:49.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b51d9b640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b540c4410 0x7f3b5415a280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009870 tx=0x7f3b4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588523b437c31113 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b5159a640 1 -- 172.21.15.8:0/1537615917 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b480ab680 con 0x7f3b540c4410 2023-10-30T12:45:49.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b5159a640 1 -- 172.21.15.8:0/1537615917 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b480ab7e0 con 0x7f3b540c4410 2023-10-30T12:45:49.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b5159a640 1 -- 172.21.15.8:0/1537615917 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b480abb70 con 0x7f3b540c4410 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- 172.21.15.8:0/1537615917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b540c4410 msgr2=0x7f3b5415a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b540c4410 0x7f3b5415a280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009870 tx=0x7f3b4801c480 comp rx=0 tx=0).stop 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- 172.21.15.8:0/1537615917 shutdown_connections 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b540c4410 0x7f3b5415a280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 --2- 172.21.15.8:0/1537615917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b54153ba0 0x7f3b540cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- 172.21.15.8:0/1537615917 >> 172.21.15.8:0/1537615917 conn(0x7f3b54071790 msgr2=0x7f3b54071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- 172.21.15.8:0/1537615917 shutdown_connections 2023-10-30T12:45:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- 172.21.15.8:0/1537615917 wait complete. 2023-10-30T12:45:49.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 Processor -- start 2023-10-30T12:45:49.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.857+0000 7f3b589c5640 1 -- start start 2023-10-30T12:45:49.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b540c4410 0x7f3b54152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b540c4810 0x7f3b540b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54152120 con 0x7f3b54153ba0 2023-10-30T12:45:49.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540b4390 con 0x7f3b540c4810 2023-10-30T12:45:49.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541450e0 con 0x7f3b540c4410 2023-10-30T12:45:49.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50330/0 (socket says 172.21.15.8:50330) 2023-10-30T12:45:49.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 -- 172.21.15.8:0/236610831 learned_addr learned my addr 172.21.15.8:0/236610831 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:49.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b5259c640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b540c4410 0x7f3b54152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b51d9b640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b540c4810 0x7f3b540b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 -- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b540c4410 msgr2=0x7f3b54152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b540c4410 0x7f3b54152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 -- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b540c4810 msgr2=0x7f3b540b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:49.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b540c4810 0x7f3b540b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:49.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 -- 172.21.15.8:0/236610831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480094d0 con 0x7f3b54153ba0 2023-10-30T12:45:49.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b52d9d640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c7c0 tx=0x7f3b4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003db0 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c005070 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c005420 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b540b3d60 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b540b3530 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.861+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5404e170 con 0x7f3b54153ba0 2023-10-30T12:45:49.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.865+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b4c005580 con 0x7f3b54153ba0 2023-10-30T12:45:49.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.865+0000 7f3b42ffd640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 0x7f3b34068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:49.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.865+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b4c083720 con 0x7f3b54153ba0 2023-10-30T12:45:49.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.865+0000 7f3b5259c640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 0x7f3b34068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:49.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.865+0000 7f3b5259c640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 0x7f3b34068990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b930 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:49.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:49.869+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b4c050b10 con 0x7f3b54153ba0 2023-10-30T12:45:50.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.049+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b540b44d0 con 0x7f3b54153ba0 2023-10-30T12:45:50.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.049+0000 7f3b42ffd640 1 -- 172.21.15.8:0/236610831 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b4c054130 con 0x7f3b54153ba0 2023-10-30T12:45:50.055 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:45:50.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 msgr2=0x7f3b34068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 0x7f3b34068990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b930 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2023-10-30T12:45:50.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 msgr2=0x7f3b541f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00c7c0 tx=0x7f3b4c00cc80 comp rx=0 tx=0).stop 2023-10-30T12:45:50.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 shutdown_connections 2023-10-30T12:45:50.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3b340664d0 0x7f3b34068990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b540c4810 0x7f3b540b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b54153ba0 0x7f3b541f56c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 --2- 172.21.15.8:0/236610831 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b540c4410 0x7f3b54152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 >> 172.21.15.8:0/236610831 conn(0x7f3b54071790 msgr2=0x7f3b540c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 shutdown_connections 2023-10-30T12:45:50.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.053+0000 7f3b589c5640 1 -- 172.21.15.8:0/236610831 wait complete. 2023-10-30T12:45:50.076 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:45:50.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 Processor -- start 2023-10-30T12:45:50.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 -- start start 2023-10-30T12:45:50.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b78153ba0 0x7f4b780cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4b780c4410 0x7f4b7815a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b78153f80 con 0x7f4b780c4810 2023-10-30T12:45:50.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b78159df0 con 0x7f4b780c4410 2023-10-30T12:45:50.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7cb01640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7815a7c0 con 0x7f4b78153ba0 2023-10-30T12:45:50.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4b780c4410 0x7f4b7815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4b780c4410 0x7f4b7815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50344/0 (socket says 172.21.15.8:50344) 2023-10-30T12:45:50.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b6ffff640 1 -- 172.21.15.8:0/3637948302 learned_addr learned my addr 172.21.15.8:0/3637948302 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:50.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.221+0000 7f4b7659c640 1 --2- 172.21.15.8:0/3637948302 >> v2:172.21.15.46:3300/0 conn(0x7f4b78153ba0 0x7f4b780cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b6ffff640 1 -- 172.21.15.8:0/3637948302 >> v2:172.21.15.46:3300/0 conn(0x7f4b78153ba0 msgr2=0x7f4b780cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:50.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b6ffff640 1 --2- 172.21.15.8:0/3637948302 >> v2:172.21.15.46:3300/0 conn(0x7f4b78153ba0 0x7f4b780cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b6ffff640 1 -- 172.21.15.8:0/3637948302 >> v1:172.21.15.46:6789/0 conn(0x7f4b780c4810 legacy=0x7f4b7815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b6ffff640 1 -- 172.21.15.8:0/3637948302 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b781597c0 con 0x7f4b780c4410 2023-10-30T12:45:50.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b6ffff640 1 --2- 172.21.15.8:0/3637948302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b7815a280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b780d0170 tx=0x7f4b6001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1e7b4363e5884f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b75d9b640 1 -- 172.21.15.8:0/3637948302 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600ab640 con 0x7f4b780c4410 2023-10-30T12:45:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b75d9b640 1 -- 172.21.15.8:0/3637948302 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b600a7070 con 0x7f4b780c4410 2023-10-30T12:45:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b75d9b640 1 -- 172.21.15.8:0/3637948302 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600ab9c0 con 0x7f4b780c4410 2023-10-30T12:45:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b7cb01640 1 -- 172.21.15.8:0/3637948302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 msgr2=0x7f4b7815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.225+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/3637948302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b7815a280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b780d0170 tx=0x7f4b6001c440 comp rx=0 tx=0).stop 2023-10-30T12:45:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- 172.21.15.8:0/3637948302 shutdown_connections 2023-10-30T12:45:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/3637948302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b7815a280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/3637948302 >> v2:172.21.15.46:3300/0 conn(0x7f4b78153ba0 0x7f4b780cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- 172.21.15.8:0/3637948302 >> 172.21.15.8:0/3637948302 conn(0x7f4b78071790 msgr2=0x7f4b78071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- 172.21.15.8:0/3637948302 shutdown_connections 2023-10-30T12:45:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- 172.21.15.8:0/3637948302 wait complete. 2023-10-30T12:45:50.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 Processor -- start 2023-10-30T12:45:50.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- start start 2023-10-30T12:45:50.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4b780c4810 0x7f4b780b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78153ba0 0x7f4b781f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78150450 con 0x7f4b780c4410 2023-10-30T12:45:50.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781f65d0 con 0x7f4b78153ba0 2023-10-30T12:45:50.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7cb01640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781f62f0 con 0x7f4b780c4810 2023-10-30T12:45:50.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50354/0 (socket says 172.21.15.8:50354) 2023-10-30T12:45:50.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 -- 172.21.15.8:0/1277254721 learned_addr learned my addr 172.21.15.8:0/1277254721 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:50.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b6ffff640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4b780c4810 0x7f4b780b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b76d9d640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78153ba0 0x7f4b781f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 -- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4b780c4810 msgr2=0x7f4b780b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4b780c4810 0x7f4b780b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 -- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78153ba0 msgr2=0x7f4b781f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78153ba0 0x7f4b781f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 -- 172.21.15.8:0/1277254721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600094d0 con 0x7f4b780c4410 2023-10-30T12:45:50.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.229+0000 7f4b7659c640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800d5f0 tx=0x7f4b6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680125f0 con 0x7f4b780c4410 2023-10-30T12:45:50.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68012750 con 0x7f4b780c4410 2023-10-30T12:45:50.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68012ae0 con 0x7f4b780c4410 2023-10-30T12:45:50.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b781512b0 con 0x7f4b780c4410 2023-10-30T12:45:50.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7815a980 con 0x7f4b780c4410 2023-10-30T12:45:50.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7804e170 con 0x7f4b780c4410 2023-10-30T12:45:50.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b68012c40 con 0x7f4b780c4410 2023-10-30T12:45:50.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 0x7f4b58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b680842b0 con 0x7f4b780c4410 2023-10-30T12:45:50.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.233+0000 7f4b6ffff640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 0x7f4b58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.237+0000 7f4b6ffff640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 0x7f4b58068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b600af000 tx=0x7f4b600b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.237+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b68051740 con 0x7f4b780c4410 2023-10-30T12:45:50.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.397+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b781523e0 con 0x7f4b780c4410 2023-10-30T12:45:50.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.397+0000 7f4b6effd640 1 -- 172.21.15.8:0/1277254721 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b68007380 con 0x7f4b780c4410 2023-10-30T12:45:50.402 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:45:50.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 msgr2=0x7f4b58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 0x7f4b58068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b600af000 tx=0x7f4b600b6040 comp rx=0 tx=0).stop 2023-10-30T12:45:50.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 msgr2=0x7f4b780b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800d5f0 tx=0x7f4b6800dab0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 shutdown_connections 2023-10-30T12:45:50.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4b580664a0 0x7f4b58068960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78153ba0 0x7f4b781f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4b780c4810 0x7f4b780b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 --2- 172.21.15.8:0/1277254721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b780c4410 0x7f4b780b3eb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 >> 172.21.15.8:0/1277254721 conn(0x7f4b78071790 msgr2=0x7f4b78147410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 shutdown_connections 2023-10-30T12:45:50.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.401+0000 7f4b7cb01640 1 -- 172.21.15.8:0/1277254721 wait complete. 2023-10-30T12:45:50.423 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:45:50.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.561+0000 7f5f25194640 1 Processor -- start 2023-10-30T12:45:50.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.561+0000 7f5f25194640 1 -- start start 2023-10-30T12:45:50.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f25194640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151c20 0x7f5f200c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f25194640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5f200cb8b0 0x7f5f2006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f25194640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200c5030 con 0x7f5f20151840 2023-10-30T12:45:50.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f25194640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f20157830 con 0x7f5f200cb8b0 2023-10-30T12:45:50.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f25194640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20157b10 con 0x7f5f20151c20 2023-10-30T12:45:50.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1f59e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5f200cb8b0 0x7f5f2006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1ed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f5f20151840 0x7f5f20072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39976/0 (socket says 172.21.15.8:39976) 2023-10-30T12:45:50.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1ed9d640 1 -- 172.21.15.8:0/2642842055 learned_addr learned my addr 172.21.15.8:0/2642842055 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:45:50.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f17fff640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151c20 0x7f5f200c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1f59e640 1 -- 172.21.15.8:0/2642842055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151c20 msgr2=0x7f5f200c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:50.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1f59e640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151c20 0x7f5f200c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1f59e640 1 -- 172.21.15.8:0/2642842055 >> v1:172.21.15.8:6789/0 conn(0x7f5f20151840 legacy=0x7f5f20072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1f59e640 1 -- 172.21.15.8:0/2642842055 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20157df0 con 0x7f5f200cb8b0 2023-10-30T12:45:50.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.565+0000 7f5f1e59c640 1 -- 172.21.15.8:0/2642842055 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614005486 0 0) 0x7f5f200c5030 con 0x7f5f20151840 2023-10-30T12:45:50.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f1f59e640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f200cb8b0 0x7f5f2006b600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009d90 tx=0x7f5f1801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672982bd0fe833b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f1e59c640 1 -- 172.21.15.8:0/2642842055 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180af600 con 0x7f5f200cb8b0 2023-10-30T12:45:50.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f1e59c640 1 -- 172.21.15.8:0/2642842055 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f180ab050 con 0x7f5f200cb8b0 2023-10-30T12:45:50.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f1e59c640 1 -- 172.21.15.8:0/2642842055 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180af980 con 0x7f5f200cb8b0 2023-10-30T12:45:50.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- 172.21.15.8:0/2642842055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f200cb8b0 msgr2=0x7f5f2006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f200cb8b0 0x7f5f2006b600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009d90 tx=0x7f5f1801f130 comp rx=0 tx=0).stop 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- 172.21.15.8:0/2642842055 shutdown_connections 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f200cb8b0 0x7f5f2006b600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 --2- 172.21.15.8:0/2642842055 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151c20 0x7f5f200c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- 172.21.15.8:0/2642842055 >> 172.21.15.8:0/2642842055 conn(0x7f5f200cac70 msgr2=0x7f5f200cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- 172.21.15.8:0/2642842055 shutdown_connections 2023-10-30T12:45:50.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- 172.21.15.8:0/2642842055 wait complete. 2023-10-30T12:45:50.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 Processor -- start 2023-10-30T12:45:50.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 -- start start 2023-10-30T12:45:50.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f200cb8b0 0x7f5f20071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.569+0000 7f5f25194640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151840 0x7f5f201eef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f201e2280 con 0x7f5f20151c20 2023-10-30T12:45:50.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f201555a0 con 0x7f5f200cb8b0 2023-10-30T12:45:50.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20155880 con 0x7f5f20151840 2023-10-30T12:45:50.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50380/0 (socket says 172.21.15.8:50380) 2023-10-30T12:45:50.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 -- 172.21.15.8:0/4177342736 learned_addr learned my addr 172.21.15.8:0/4177342736 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:50.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f17fff640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151840 0x7f5f201eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1ed9d640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f200cb8b0 0x7f5f20071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 -- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151840 msgr2=0x7f5f201eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151840 0x7f5f201eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 -- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f200cb8b0 msgr2=0x7f5f20071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f200cb8b0 0x7f5f20071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 -- 172.21.15.8:0/4177342736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009a70 con 0x7f5f20151c20 2023-10-30T12:45:50.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f1f59e640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f1801d7e0 tx=0x7f5f1801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180b9610 con 0x7f5f20151c20 2023-10-30T12:45:50.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f180b9770 con 0x7f5f20151c20 2023-10-30T12:45:50.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180b9b00 con 0x7f5f20151c20 2023-10-30T12:45:50.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f20155b60 con 0x7f5f20151c20 2023-10-30T12:45:50.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f201e1350 con 0x7f5f20151c20 2023-10-30T12:45:50.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f20066510 con 0x7f5f20151c20 2023-10-30T12:45:50.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.573+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5f180b6050 con 0x7f5f20151c20 2023-10-30T12:45:50.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.577+0000 7f5f177fe640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 0x7f5f00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.577+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f181224d0 con 0x7f5f20151c20 2023-10-30T12:45:50.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.577+0000 7f5f1ed9d640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 0x7f5f00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.577+0000 7f5f1ed9d640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 0x7f5f00068960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f70 tx=0x7f5f10007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.581+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5f180ef9e0 con 0x7f5f20151c20 2023-10-30T12:45:50.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.765+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f20156010 con 0x7f5f20151c20 2023-10-30T12:45:50.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.765+0000 7f5f177fe640 1 -- 172.21.15.8:0/4177342736 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f5f18009450 con 0x7f5f20151c20 2023-10-30T12:45:50.770 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:50.770 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:45:50.771 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:45:50.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 msgr2=0x7f5f00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 0x7f5f00068960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f70 tx=0x7f5f10007b60 comp rx=0 tx=0).stop 2023-10-30T12:45:50.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 msgr2=0x7f5f201ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f1801d7e0 tx=0x7f5f1801bed0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 shutdown_connections 2023-10-30T12:45:50.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5f000664a0 0x7f5f00068960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5f20151c20 0x7f5f201ef440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5f20151840 0x7f5f201eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 --2- 172.21.15.8:0/4177342736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f200cb8b0 0x7f5f20071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 >> 172.21.15.8:0/4177342736 conn(0x7f5f200cac70 msgr2=0x7f5f200c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 shutdown_connections 2023-10-30T12:45:50.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.769+0000 7f5f25194640 1 -- 172.21.15.8:0/4177342736 wait complete. 2023-10-30T12:45:50.791 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:45:50.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.929+0000 7f459fc17640 1 Processor -- start 2023-10-30T12:45:50.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.929+0000 7f459fc17640 1 -- start start 2023-10-30T12:45:50.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459fc17640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f45981516c0 0x7f45980c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459fc17640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45980c82d0 0x7f45980ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459fc17640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4598151aa0 con 0x7f45980cd4a0 2023-10-30T12:45:50.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459fc17640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45980c74a0 con 0x7f45981516c0 2023-10-30T12:45:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459fc17640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4598158510 con 0x7f45980c82d0 2023-10-30T12:45:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f45981516c0 0x7f45980c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f45981516c0 0x7f45980c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50388/0 (socket says 172.21.15.8:50388) 2023-10-30T12:45:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 -- 172.21.15.8:0/4006945201 learned_addr learned my addr 172.21.15.8:0/4006945201 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d1b2640 1 --2- 172.21.15.8:0/4006945201 >> v2:172.21.15.46:3300/0 conn(0x7f45980c82d0 0x7f45980ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 -- 172.21.15.8:0/4006945201 >> v2:172.21.15.46:3300/0 conn(0x7f45980c82d0 msgr2=0x7f45980ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:45:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 --2- 172.21.15.8:0/4006945201 >> v2:172.21.15.46:3300/0 conn(0x7f45980c82d0 0x7f45980ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 -- 172.21.15.8:0/4006945201 >> v1:172.21.15.46:6789/0 conn(0x7f45980cd4a0 legacy=0x7f45980c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.933+0000 7f459d9b3640 1 -- 172.21.15.8:0/4006945201 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598158af0 con 0x7f45981516c0 2023-10-30T12:45:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459d9b3640 1 --2- 172.21.15.8:0/4006945201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45980c7d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45980caae0 tx=0x7f459401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0b96f2c99b8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459c9b1640 1 -- 172.21.15.8:0/4006945201 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940ab610 con 0x7f45981516c0 2023-10-30T12:45:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459c9b1640 1 -- 172.21.15.8:0/4006945201 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45940ab770 con 0x7f45981516c0 2023-10-30T12:45:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459c9b1640 1 -- 172.21.15.8:0/4006945201 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940abb00 con 0x7f45981516c0 2023-10-30T12:45:50.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- 172.21.15.8:0/4006945201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 msgr2=0x7f45980c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- 172.21.15.8:0/4006945201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45980c7d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45980caae0 tx=0x7f459401c500 comp rx=0 tx=0).stop 2023-10-30T12:45:50.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- 172.21.15.8:0/4006945201 shutdown_connections 2023-10-30T12:45:50.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- 172.21.15.8:0/4006945201 >> v2:172.21.15.46:3300/0 conn(0x7f45980c82d0 0x7f45980ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- 172.21.15.8:0/4006945201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45980c7d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- 172.21.15.8:0/4006945201 >> 172.21.15.8:0/4006945201 conn(0x7f4598071fb0 msgr2=0x7f45980723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:50.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- 172.21.15.8:0/4006945201 shutdown_connections 2023-10-30T12:45:50.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- 172.21.15.8:0/4006945201 wait complete. 2023-10-30T12:45:50.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 Processor -- start 2023-10-30T12:45:50.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 -- start start 2023-10-30T12:45:50.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45980c82d0 0x7f45980c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f45980cd4a0 0x7f45981f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.937+0000 7f459fc17640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980cba00 con 0x7f45981516c0 2023-10-30T12:45:50.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459815a6f0 con 0x7f45980c82d0 2023-10-30T12:45:50.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45980ccc60 con 0x7f45980cd4a0 2023-10-30T12:45:50.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50398/0 (socket says 172.21.15.8:50398) 2023-10-30T12:45:50.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459d1b2640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f45980cd4a0 0x7f45981f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 -- 172.21.15.8:0/757269770 learned_addr learned my addr 172.21.15.8:0/757269770 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459d9b3640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45980c82d0 0x7f45980c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 -- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f45980cd4a0 msgr2=0x7f45981f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f45980cd4a0 0x7f45981f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 -- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45980c82d0 msgr2=0x7f45980c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:50.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45980c82d0 0x7f45980c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:50.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 -- 172.21.15.8:0/757269770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940094d0 con 0x7f45981516c0 2023-10-30T12:45:50.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459e1b4640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f459000c760 tx=0x7f459000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590003b80 con 0x7f45981516c0 2023-10-30T12:45:50.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4590003ce0 con 0x7f45981516c0 2023-10-30T12:45:50.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459000aa30 con 0x7f45981516c0 2023-10-30T12:45:50.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45980cc650 con 0x7f45981516c0 2023-10-30T12:45:50.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45980cc8f0 con 0x7f45981516c0 2023-10-30T12:45:50.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459804e170 con 0x7f45981516c0 2023-10-30T12:45:50.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.941+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f459000ab90 con 0x7f45981516c0 2023-10-30T12:45:50.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.945+0000 7f458e7fc640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 0x7f457c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:45:50.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.945+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4590083b70 con 0x7f45981516c0 2023-10-30T12:45:50.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.945+0000 7f459d9b3640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 0x7f457c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:45:50.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.945+0000 7f459d9b3640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 0x7f457c068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45940af000 tx=0x7f45940b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:45:50.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:50.949+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4590051000 con 0x7f45981516c0 2023-10-30T12:45:51.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.109+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4598154f20 con 0x7f45981516c0 2023-10-30T12:45:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.109+0000 7f458e7fc640 1 -- 172.21.15.8:0/757269770 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16385 (secure 0 0 0) 0x7f4590054620 con 0x7f45981516c0 2023-10-30T12:45:51.114 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:45:51.115 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:45:20.062415+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:45:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 msgr2=0x7f457c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 0x7f457c068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45940af000 tx=0x7f45940b6040 comp rx=0 tx=0).stop 2023-10-30T12:45:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 msgr2=0x7f45981f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:45:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f459000c760 tx=0x7f459000cc20 comp rx=0 tx=0).stop 2023-10-30T12:45:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 shutdown_connections 2023-10-30T12:45:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f457c0664a0 0x7f457c068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45981516c0 0x7f45981f2500 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f45980cd4a0 0x7f45981f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 --2- 172.21.15.8:0/757269770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45980c82d0 0x7f45980c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:45:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 >> 172.21.15.8:0/757269770 conn(0x7f4598071fb0 msgr2=0x7f459814f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:45:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 shutdown_connections 2023-10-30T12:45:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:45:51.113+0000 7f459fc17640 1 -- 172.21.15.8:0/757269770 wait complete. 2023-10-30T12:45:51.136 DEBUG:tasks.mds_thrash:Yielding 2023-10-30T12:45:51.137 INFO:teuthology.run_tasks:Running task workunit... 2023-10-30T12:45:51.149 INFO:tasks.workunit:Pulling workunits from ref 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:45:51.150 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-30T12:45:51.150 INFO:tasks.workunit:timeout=6h 2023-10-30T12:45:51.151 INFO:tasks.workunit:cleanup=True 2023-10-30T12:45:51.151 DEBUG:teuthology.orchestra.run.smithi008:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:51.151 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2023-10-30T12:45:51.152 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 75 secs before thrashing 2023-10-30T12:45:51.161 INFO:teuthology.orchestra.run.smithi008.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:51.161 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-30T12:45:51.161 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 2bh/43d Inode: 1 Links: 2 2023-10-30T12:45:51.161 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T12:45:51.162 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-10-30T12:45:51.162 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-30 12:45:24.067022976 +0000 2023-10-30T12:45:51.162 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-30 12:45:43.489287499 +0000 2023-10-30T12:45:51.162 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-10-30 12:45:24.067022976 +0000 2023-10-30T12:45:51.162 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-30T12:45:51.163 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-30T12:45:51.230 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 00c1c13cea21a686b046584596fbeac5315a7d71 2023-10-30T12:45:51.285 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-30T12:46:14.013 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:14.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:44.073 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:44.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:46:55.656 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 64% (7822/12071) Updating files: 65% (7847/12071) Updating files: 66% (7967/12071) Updating files: 67% (8088/12071) Updating files: 68% (8209/12071) Updating files: 69% (8329/12071) Updating files: 70% (8450/12071) Updating files: 71% (8571/12071) Updating files: 72% (8692/12071) Updating files: 73% (8812/12071) Updating files: 74% (8933/12071) Updating files: 75% (9054/12071) Updating files: 76% (9174/12071) Updating files: 77% (9295/12071) Updating files: 78% (9416/12071) Updating files: 79% (9537/12071) Updating files: 80% (9657/12071) Updating files: 81% (9778/12071) Updating files: 82% (9899/12071) Updating files: 83% (10019/12071) Updating files: 84% (10140/12071) Updating files: 85% (10261/12071) Updating files: 86% (10382/12071) Updating files: 87% (10502/12071) Updating files: 88% (10623/12071) Updating files: 89% (10744/12071) Updating files: 90% (10864/12071) Updating files: 91% (10985/12071) Updating files: 92% (11106/12071) Updating files: 93% (11227/12071) Updating files: 94% (11347/12071) Updating files: 95% (11468/12071) Updating files: 96% (11589/12071) Updating files: 97% (11709/12071) Updating files: 98% (11830/12071) Updating files: 99% (11951/12071) Updating files: 100% (12071/12071) Updating files: 100% (12071/12071), done. 2023-10-30T12:46:56.194 INFO:tasks.workunit.client.0.smithi008.stderr:Note: switching to '00c1c13cea21a686b046584596fbeac5315a7d71'. 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr:state without impacting any branches by switching back to a branch. 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.195 INFO:tasks.workunit.client.0.smithi008.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-30T12:46:56.196 INFO:tasks.workunit.client.0.smithi008.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-30T12:46:56.196 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.196 INFO:tasks.workunit.client.0.smithi008.stderr: git switch -c 2023-10-30T12:46:56.196 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.196 INFO:tasks.workunit.client.0.smithi008.stderr:Or undo this operation with: 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr: git switch - 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-10-30T12:46:56.197 INFO:tasks.workunit.client.0.smithi008.stderr:HEAD is now at 00c1c13cea2 mds/MDBalancer: ignore queued callbacks if MDS is not active 2023-10-30T12:46:56.200 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-30T12:46:56.210 INFO:tasks.workunit.client.0.smithi008.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-30T12:46:56.213 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-30T12:46:56.213 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-30T12:46:56.940 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-30T12:46:57.022 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-30T12:46:57.084 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-30T12:46:57.087 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-30T12:46:57.087 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-30T12:46:57.142 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-30T12:46:57.147 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:46:57.148 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-30T12:46:57.203 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2023-10-30T12:46:57.205 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2023-10-30T12:46:57.205 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2023-10-30T12:46:57.264 INFO:tasks.workunit.client.0.smithi008.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2023-10-30T12:46:59.608 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir foo 2023-10-30T12:46:59.618 INFO:tasks.workunit.client.0.smithi008.stderr:+ cp linux-2.6.33.tar.bz2 foo 2023-10-30T12:46:59.882 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir foo/.snap/barsnap 2023-10-30T12:46:59.895 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm foo/linux-2.6.33.tar.bz2 2023-10-30T12:46:59.906 INFO:tasks.workunit.client.0.smithi008.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2023-10-30T12:47:00.839 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo passed: files are identical 2023-10-30T12:47:00.839 INFO:tasks.workunit.client.0.smithi008.stdout:passed: files are identical 2023-10-30T12:47:00.840 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir foo/.snap/barsnap 2023-10-30T12:47:00.871 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:47:00.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:47:00.872 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:47:00.872 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:47:00.953 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2023-10-30T12:47:00.954 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2023-10-30T12:47:01.018 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 1 2023-10-30T12:47:01.019 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 2 2023-10-30T12:47:01.020 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-10-30T12:47:01.020 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' -e file4 ']' 2023-10-30T12:47:01.020 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir .snap/snap1 2023-10-30T12:47:01.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 4 2023-10-30T12:47:01.039 INFO:tasks.workunit.client.0.smithi008.stderr:++ ls 2023-10-30T12:47:01.045 INFO:tasks.workunit.client.0.smithi008.stderr:+ now='file1 2023-10-30T12:47:01.045 INFO:tasks.workunit.client.0.smithi008.stderr:file2 2023-10-30T12:47:01.045 INFO:tasks.workunit.client.0.smithi008.stderr:file3 2023-10-30T12:47:01.046 INFO:tasks.workunit.client.0.smithi008.stderr:file4' 2023-10-30T12:47:01.046 INFO:tasks.workunit.client.0.smithi008.stderr:++ ls .snap/snap1 2023-10-30T12:47:01.050 INFO:tasks.workunit.client.0.smithi008.stderr:+ then='file1 2023-10-30T12:47:01.051 INFO:tasks.workunit.client.0.smithi008.stderr:file2 2023-10-30T12:47:01.051 INFO:tasks.workunit.client.0.smithi008.stderr:file3' 2023-10-30T12:47:01.051 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir .snap/snap1 2023-10-30T12:47:01.060 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 'file1 2023-10-30T12:47:01.061 INFO:tasks.workunit.client.0.smithi008.stderr:file2 2023-10-30T12:47:01.061 INFO:tasks.workunit.client.0.smithi008.stderr:file3 2023-10-30T12:47:01.061 INFO:tasks.workunit.client.0.smithi008.stderr:file4' = 'file1 2023-10-30T12:47:01.061 INFO:tasks.workunit.client.0.smithi008.stderr:file2 2023-10-30T12:47:01.061 INFO:tasks.workunit.client.0.smithi008.stderr:file3' ']' 2023-10-30T12:47:01.062 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 1 2023-10-30T12:47:01.070 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 2 2023-10-30T12:47:01.080 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-10-30T12:47:01.089 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir .snap/snap1 2023-10-30T12:47:01.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 4 2023-10-30T12:47:01.109 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir .snap/snap1 2023-10-30T12:47:01.118 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm file1 file2 file3 file4 2023-10-30T12:47:01.125 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:47:01.126 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:47:01.127 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:47:01.127 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:47:01.196 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2023-10-30T12:47:01.196 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2023-10-30T12:47:01.262 INFO:tasks.workunit.client.0.smithi008.stdout:Create dir 100 to 199 ... 2023-10-30T12:47:01.264 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 100 2023-10-30T12:47:01.270 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 101 2023-10-30T12:47:01.279 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 102 2023-10-30T12:47:02.073 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 103 2023-10-30T12:47:02.268 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 104 2023-10-30T12:47:02.279 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 105 2023-10-30T12:47:02.290 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 106 2023-10-30T12:47:02.301 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 107 2023-10-30T12:47:02.312 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 108 2023-10-30T12:47:02.322 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 109 2023-10-30T12:47:02.332 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 110 2023-10-30T12:47:02.841 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 111 2023-10-30T12:47:02.857 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 112 2023-10-30T12:47:02.872 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 113 2023-10-30T12:47:02.886 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 114 2023-10-30T12:47:02.900 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 115 2023-10-30T12:47:02.915 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 116 2023-10-30T12:47:02.929 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 117 2023-10-30T12:47:02.943 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 118 2023-10-30T12:47:03.065 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 119 2023-10-30T12:47:03.459 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 120 2023-10-30T12:47:04.057 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 121 2023-10-30T12:47:04.251 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 122 2023-10-30T12:47:04.261 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 123 2023-10-30T12:47:04.270 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 124 2023-10-30T12:47:04.279 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 125 2023-10-30T12:47:04.289 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 126 2023-10-30T12:47:04.298 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 127 2023-10-30T12:47:04.308 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 128 2023-10-30T12:47:04.317 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 129 2023-10-30T12:47:04.749 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 130 2023-10-30T12:47:04.768 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 131 2023-10-30T12:47:04.778 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 132 2023-10-30T12:47:04.788 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 133 2023-10-30T12:47:04.797 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 134 2023-10-30T12:47:04.806 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 135 2023-10-30T12:47:04.815 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 136 2023-10-30T12:47:04.824 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 137 2023-10-30T12:47:04.833 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 138 2023-10-30T12:47:04.842 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 139 2023-10-30T12:47:04.851 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 140 2023-10-30T12:47:04.860 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 141 2023-10-30T12:47:04.868 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 142 2023-10-30T12:47:04.877 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 143 2023-10-30T12:47:04.886 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 144 2023-10-30T12:47:04.895 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 145 2023-10-30T12:47:04.904 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 146 2023-10-30T12:47:04.913 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 147 2023-10-30T12:47:04.922 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 148 2023-10-30T12:47:05.512 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 149 2023-10-30T12:47:05.718 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 150 2023-10-30T12:47:05.728 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 151 2023-10-30T12:47:05.738 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 152 2023-10-30T12:47:06.073 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 153 2023-10-30T12:47:06.152 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:06.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 Processor -- start 2023-10-30T12:47:06.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 -- start start 2023-10-30T12:47:06.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efc5806c420 0x7efc580c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc580c5f80 con 0x7efc58153bb0 2023-10-30T12:47:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc580c5690 con 0x7efc5806c420 2023-10-30T12:47:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc5d30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5815a580 con 0x7efc580c7170 2023-10-30T12:47:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc4ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efc5806c420 0x7efc580c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc4ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efc5806c420 0x7efc580c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34476/0 (socket says 172.21.15.8:34476) 2023-10-30T12:47:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.291+0000 7efc4ffff640 1 -- 172.21.15.8:0/2891264896 learned_addr learned my addr 172.21.15.8:0/2891264896 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc577fe640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc4ffff640 1 -- 172.21.15.8:0/2891264896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 msgr2=0x7efc58159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc4ffff640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc4ffff640 1 -- 172.21.15.8:0/2891264896 >> v1:172.21.15.8:6789/0 conn(0x7efc58153bb0 legacy=0x7efc580c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc4ffff640 1 -- 172.21.15.8:0/2891264896 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580c5930 con 0x7efc5806c420 2023-10-30T12:47:06.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc567fc640 1 -- 172.21.15.8:0/2891264896 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923244490 0 0) 0x7efc580c5f80 con 0x7efc58153bb0 2023-10-30T12:47:06.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc4ffff640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc580c6c30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efc40009870 tx=0x7efc4001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8951eb629d9293 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc567fc640 1 -- 172.21.15.8:0/2891264896 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400a8a30 con 0x7efc5806c420 2023-10-30T12:47:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc567fc640 1 -- 172.21.15.8:0/2891264896 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc400a8b90 con 0x7efc5806c420 2023-10-30T12:47:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.295+0000 7efc567fc640 1 -- 172.21.15.8:0/2891264896 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400b0440 con 0x7efc5806c420 2023-10-30T12:47:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- 172.21.15.8:0/2891264896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 msgr2=0x7efc580c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc580c6c30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efc40009870 tx=0x7efc4001ed30 comp rx=0 tx=0).stop 2023-10-30T12:47:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- 172.21.15.8:0/2891264896 shutdown_connections 2023-10-30T12:47:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- 172.21.15.8:0/2891264896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc580c6c30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- 172.21.15.8:0/2891264896 >> 172.21.15.8:0/2891264896 conn(0x7efc580cac70 msgr2=0x7efc580cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- 172.21.15.8:0/2891264896 shutdown_connections 2023-10-30T12:47:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- 172.21.15.8:0/2891264896 wait complete. 2023-10-30T12:47:06.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 Processor -- start 2023-10-30T12:47:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- start start 2023-10-30T12:47:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc58153bb0 0x7efc581eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581f2940 con 0x7efc5806c420 2023-10-30T12:47:06.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc580c3d50 con 0x7efc58153bb0 2023-10-30T12:47:06.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc5d30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc581ee7d0 con 0x7efc580c7170 2023-10-30T12:47:06.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34492/0 (socket says 172.21.15.8:34492) 2023-10-30T12:47:06.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 -- 172.21.15.8:0/3778792335 learned_addr learned my addr 172.21.15.8:0/3778792335 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:06.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc577fe640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc58153bb0 0x7efc581eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc4ffff640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 -- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 msgr2=0x7efc58151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 -- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc58153bb0 msgr2=0x7efc581eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc58153bb0 0x7efc581eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 -- 172.21.15.8:0/3778792335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400094d0 con 0x7efc5806c420 2023-10-30T12:47:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc4ffff640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.299+0000 7efc56ffd640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c450 tx=0x7efc4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48005480 con 0x7efc5806c420 2023-10-30T12:47:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc48003600 con 0x7efc5806c420 2023-10-30T12:47:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc581573b0 con 0x7efc5806c420 2023-10-30T12:47:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480039b0 con 0x7efc5806c420 2023-10-30T12:47:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc580c3fa0 con 0x7efc5806c420 2023-10-30T12:47:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc58070150 con 0x7efc5806c420 2023-10-30T12:47:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc48003b10 con 0x7efc5806c420 2023-10-30T12:47:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 0x7efc38068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7efc480835a0 con 0x7efc5806c420 2023-10-30T12:47:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4ffff640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 0x7efc38068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.303+0000 7efc4ffff640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 0x7efc38068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc400b2000 tx=0x7efc400aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.307+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc48050820 con 0x7efc5806c420 2023-10-30T12:47:06.445 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 154 2023-10-30T12:47:06.461 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 155 2023-10-30T12:47:06.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.515+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc5804e170 con 0x7efc5806c420 2023-10-30T12:47:06.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc4f7fe640 1 -- 172.21.15.8:0/3778792335 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7efc48017020 con 0x7efc5806c420 2023-10-30T12:47:06.524 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:06.524 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:06.525 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:06.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 msgr2=0x7efc38068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 0x7efc38068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc400b2000 tx=0x7efc400aa070 comp rx=0 tx=0).stop 2023-10-30T12:47:06.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 msgr2=0x7efc581f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c450 tx=0x7efc4800c910 comp rx=0 tx=0).stop 2023-10-30T12:47:06.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 shutdown_connections 2023-10-30T12:47:06.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc380664a0 0x7efc38068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc58153bb0 0x7efc581eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc580c7170 0x7efc58151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 --2- 172.21.15.8:0/3778792335 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc5806c420 0x7efc581f2400 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 >> 172.21.15.8:0/3778792335 conn(0x7efc580cac70 msgr2=0x7efc5806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 shutdown_connections 2023-10-30T12:47:06.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.519+0000 7efc5d30f640 1 -- 172.21.15.8:0/3778792335 wait complete. 2023-10-30T12:47:06.545 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2023-10-30T12:47:06.545 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:47:06.546 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:47:06.559 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:47:06.559 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:06.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 Processor -- start 2023-10-30T12:47:06.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 -- start start 2023-10-30T12:47:06.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18400c4a10 0x7f184015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f18400c42a0 0x7f18400ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1840159f20 con 0x7f1840153c80 2023-10-30T12:47:06.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184015dec0 con 0x7f18400c4a10 2023-10-30T12:47:06.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1846d3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18401599a0 con 0x7f18400c42a0 2023-10-30T12:47:06.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f18452db640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f18400c42a0 0x7f18400ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f183ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18400c4a10 0x7f184015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1844ada640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1840153c80 0x7f184015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:45452/0 (socket says 172.21.15.8:45452) 2023-10-30T12:47:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f1844ada640 1 -- 172.21.15.8:0/209120490 learned_addr learned my addr 172.21.15.8:0/209120490 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f183ffff640 1 -- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f18400c42a0 msgr2=0x7f18400ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f183ffff640 1 --2- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f18400c42a0 0x7f18400ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f183ffff640 1 -- 172.21.15.8:0/209120490 >> v1:172.21.15.46:6789/0 conn(0x7f1840153c80 legacy=0x7f184015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.739+0000 7f183ffff640 1 -- 172.21.15.8:0/209120490 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184015d620 con 0x7f18400c4a10 2023-10-30T12:47:06.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.743+0000 7f183ffff640 1 --2- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c4a10 0x7f184015cf90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f182c0097c0 tx=0x7f182c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13854db9db4efb71 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.743+0000 7f183f7fe640 1 -- 172.21.15.8:0/209120490 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0a8970 con 0x7f18400c4a10 2023-10-30T12:47:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.743+0000 7f183f7fe640 1 -- 172.21.15.8:0/209120490 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c0a8ad0 con 0x7f18400c4a10 2023-10-30T12:47:06.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.743+0000 7f183f7fe640 1 -- 172.21.15.8:0/209120490 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0a8e60 con 0x7f18400c4a10 2023-10-30T12:47:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c4a10 msgr2=0x7f184015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c4a10 0x7f184015cf90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f182c0097c0 tx=0x7f182c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:47:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- 172.21.15.8:0/209120490 shutdown_connections 2023-10-30T12:47:06.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f18400c42a0 0x7f18400ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- 172.21.15.8:0/209120490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c4a10 0x7f184015cf90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- 172.21.15.8:0/209120490 >> 172.21.15.8:0/209120490 conn(0x7f18400715c0 msgr2=0x7f18400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- 172.21.15.8:0/209120490 shutdown_connections 2023-10-30T12:47:06.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- 172.21.15.8:0/209120490 wait complete. 2023-10-30T12:47:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 Processor -- start 2023-10-30T12:47:06.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- start start 2023-10-30T12:47:06.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c42a0 0x7f184014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1840153c80 0x7f184015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18401550d0 con 0x7f18400c4a10 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184014ba50 con 0x7f18400c42a0 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1846d3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1840152670 con 0x7f1840153c80 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f183ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f183ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34502/0 (socket says 172.21.15.8:34502) 2023-10-30T12:47:06.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f183ffff640 1 -- 172.21.15.8:0/1482625491 learned_addr learned my addr 172.21.15.8:0/1482625491 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:06.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f18452db640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1840153c80 0x7f184015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.747+0000 7f1844ada640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c42a0 0x7f184014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 -- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1840153c80 msgr2=0x7f184015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1840153c80 0x7f184015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 -- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c42a0 msgr2=0x7f184014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c42a0 0x7f184014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 -- 172.21.15.8:0/1482625491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c009420 con 0x7f18400c4a10 2023-10-30T12:47:06.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183ffff640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b3040 tx=0x7f182c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0b9630 con 0x7f18400c4a10 2023-10-30T12:47:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c0b97b0 con 0x7f18400c4a10 2023-10-30T12:47:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0b9b60 con 0x7f18400c4a10 2023-10-30T12:47:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18401520b0 con 0x7f18400c4a10 2023-10-30T12:47:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184014aa30 con 0x7f18400c4a10 2023-10-30T12:47:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184004e170 con 0x7f18400c4a10 2023-10-30T12:47:06.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f182c0b7040 con 0x7f18400c4a10 2023-10-30T12:47:06.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 0x7f1824068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:06.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.751+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f182c121ee0 con 0x7f18400c4a10 2023-10-30T12:47:06.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.755+0000 7f1844ada640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 0x7f1824068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:06.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.755+0000 7f1844ada640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 0x7f1824068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1834005890 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.755+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f182c0ef330 con 0x7f18400c4a10 2023-10-30T12:47:06.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.951+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18400cad40 con 0x7f18400c4a10 2023-10-30T12:47:06.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f183d7fa640 1 -- 172.21.15.8:0/1482625491 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f182c0b5030 con 0x7f18400c4a10 2023-10-30T12:47:06.960 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:06.960 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:06.960 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:06.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 msgr2=0x7f1824068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 0x7f1824068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1834005890 tx=0x7f1834009040 comp rx=0 tx=0).stop 2023-10-30T12:47:06.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 msgr2=0x7f1840149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:06.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b3040 tx=0x7f182c01ba10 comp rx=0 tx=0).stop 2023-10-30T12:47:06.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 shutdown_connections 2023-10-30T12:47:06.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f18240664a0 0x7f1824068960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1840153c80 0x7f184015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18400c4a10 0x7f1840149ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 --2- 172.21.15.8:0/1482625491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18400c42a0 0x7f184014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:06.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 >> 172.21.15.8:0/1482625491 conn(0x7f18400715c0 msgr2=0x7f18400c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:06.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 shutdown_connections 2023-10-30T12:47:06.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:06.955+0000 7f1846d3e640 1 -- 172.21.15.8:0/1482625491 wait complete. 2023-10-30T12:47:06.981 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:07.073 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 156 2023-10-30T12:47:07.129 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 157 2023-10-30T12:47:07.340 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 158 2023-10-30T12:47:07.350 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 159 2023-10-30T12:47:07.360 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 160 2023-10-30T12:47:08.057 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 161 2023-10-30T12:47:08.126 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 162 2023-10-30T12:47:08.142 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 163 2023-10-30T12:47:08.157 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 164 2023-10-30T12:47:08.169 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 165 2023-10-30T12:47:08.946 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 166 2023-10-30T12:47:08.958 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 167 2023-10-30T12:47:08.983 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:09.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.127+0000 7fe5428ea640 1 Processor -- start 2023-10-30T12:47:09.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.127+0000 7fe5428ea640 1 -- start start 2023-10-30T12:47:09.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe53c1516c0 0x7fe53c0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0c82d0 0x7fe53c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 0x7fe53c0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c0c74a0 con 0x7fe53c1516c0 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c151aa0 con 0x7fe53c0cd4a0 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe5428ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c158af0 con 0x7fe53c0c82d0 2023-10-30T12:47:09.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe53c1516c0 0x7fe53c0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe53c1516c0 0x7fe53c0c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37740/0 (socket says 172.21.15.8:37740) 2023-10-30T12:47:09.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 -- 172.21.15.8:0/1033568606 learned_addr learned my addr 172.21.15.8:0/1033568606 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:09.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53b7fe640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0c82d0 0x7fe53c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe540e87640 1 --2- 172.21.15.8:0/1033568606 >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 0x7fe53c0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 -- 172.21.15.8:0/1033568606 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0c82d0 msgr2=0x7fe53c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:09.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0c82d0 0x7fe53c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 -- 172.21.15.8:0/1033568606 >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 msgr2=0x7fe53c0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:09.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 --2- 172.21.15.8:0/1033568606 >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 0x7fe53c0c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe53bfff640 1 -- 172.21.15.8:0/1033568606 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c158510 con 0x7fe53c1516c0 2023-10-30T12:47:09.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.131+0000 7fe540e87640 1 --2- 172.21.15.8:0/1033568606 >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 0x7fe53c0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:09.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe53bfff640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c0c7d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe53c071f60 tx=0x7fe52c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d81162035d9517 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe53affd640 1 -- 172.21.15.8:0/1033568606 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0ab600 con 0x7fe53c1516c0 2023-10-30T12:47:09.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe53affd640 1 -- 172.21.15.8:0/1033568606 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c0a7050 con 0x7fe53c1516c0 2023-10-30T12:47:09.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe53affd640 1 -- 172.21.15.8:0/1033568606 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0ab980 con 0x7fe53c1516c0 2023-10-30T12:47:09.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 -- 172.21.15.8:0/1033568606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 msgr2=0x7fe53c0c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c0c7d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe53c071f60 tx=0x7fe52c01c500 comp rx=0 tx=0).stop 2023-10-30T12:47:09.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 -- 172.21.15.8:0/1033568606 shutdown_connections 2023-10-30T12:47:09.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 --2- 172.21.15.8:0/1033568606 >> v2:172.21.15.46:3300/0 conn(0x7fe53c0cd4a0 0x7fe53c0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0c82d0 0x7fe53c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 --2- 172.21.15.8:0/1033568606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c0c7d90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 -- 172.21.15.8:0/1033568606 >> 172.21.15.8:0/1033568606 conn(0x7fe53c071fb0 msgr2=0x7fe53c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:09.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 -- 172.21.15.8:0/1033568606 shutdown_connections 2023-10-30T12:47:09.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 -- 172.21.15.8:0/1033568606 wait complete. 2023-10-30T12:47:09.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.135+0000 7fe5428ea640 1 Processor -- start 2023-10-30T12:47:09.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- start start 2023-10-30T12:47:09.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53c0c82d0 0x7fe53c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 0x7fe53c1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1f3600 con 0x7fe53c1516c0 2023-10-30T12:47:09.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0ca060 con 0x7fe53c0c82d0 2023-10-30T12:47:09.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0c5080 con 0x7fe53c0cd4a0 2023-10-30T12:47:09.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37746/0 (socket says 172.21.15.8:37746) 2023-10-30T12:47:09.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 -- 172.21.15.8:0/2605373384 learned_addr learned my addr 172.21.15.8:0/2605373384 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:09.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe53bfff640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53c0c82d0 0x7fe53c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe53b7fe640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 0x7fe53c1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 -- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 msgr2=0x7fe53c1f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 0x7fe53c1f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 -- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53c0c82d0 msgr2=0x7fe53c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53c0c82d0 0x7fe53c147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 -- 172.21.15.8:0/2605373384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c0094d0 con 0x7fe53c1516c0 2023-10-30T12:47:09.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe53b7fe640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 0x7fe53c1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:09.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe540e87640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe53400cd50 tx=0x7fe53400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400f680 con 0x7fe53c1516c0 2023-10-30T12:47:09.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400f800 con 0x7fe53c1516c0 2023-10-30T12:47:09.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c0c6310 con 0x7fe53c1516c0 2023-10-30T12:47:09.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400fbb0 con 0x7fe53c1516c0 2023-10-30T12:47:09.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.139+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c0ca2b0 con 0x7fe53c1516c0 2023-10-30T12:47:09.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.143+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c1f3740 con 0x7fe53c1516c0 2023-10-30T12:47:09.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.143+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe53400b040 con 0x7fe53c1516c0 2023-10-30T12:47:09.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.143+0000 7fe538ff9640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 0x7fe528068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:09.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.143+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fe534084960 con 0x7fe53c1516c0 2023-10-30T12:47:09.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.143+0000 7fe53bfff640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 0x7fe528068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:09.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.147+0000 7fe53bfff640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 0x7fe528068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ad000 tx=0x7fe52c0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:09.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.151+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe53c1f3740 con 0x7fe53c1516c0 2023-10-30T12:47:09.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe53c1f3740 con 0x7fe53c1516c0 2023-10-30T12:47:09.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe538ff9640 1 -- 172.21.15.8:0/2605373384 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7fe53c1f3740 con 0x7fe53c1516c0 2023-10-30T12:47:09.352 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:09.353 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:09.353 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:09.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 msgr2=0x7fe528068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 0x7fe528068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ad000 tx=0x7fe52c0a6000 comp rx=0 tx=0).stop 2023-10-30T12:47:09.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 msgr2=0x7fe53c1f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:09.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe53400cd50 tx=0x7fe53400a650 comp rx=0 tx=0).stop 2023-10-30T12:47:09.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 shutdown_connections 2023-10-30T12:47:09.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5280664a0 0x7fe528068960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe53c1516c0 0x7fe53c1f3f60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe53c0cd4a0 0x7fe53c1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 --2- 172.21.15.8:0/2605373384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe53c0c82d0 0x7fe53c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:09.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 >> 172.21.15.8:0/2605373384 conn(0x7fe53c071fb0 msgr2=0x7fe53c072390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:09.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 shutdown_connections 2023-10-30T12:47:09.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:09.347+0000 7fe5428ea640 1 -- 172.21.15.8:0/2605373384 wait complete. 2023-10-30T12:47:09.373 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:10.086 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 168 2023-10-30T12:47:11.375 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:11.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 Processor -- start 2023-10-30T12:47:11.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 -- start start 2023-10-30T12:47:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa0600c4a10 0x7fa06015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa0600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa060159f20 con 0x7fa060153c80 2023-10-30T12:47:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06015dec0 con 0x7fa0600c4a10 2023-10-30T12:47:11.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.523+0000 7fa065017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0601599a0 con 0x7fa0600c42a0 2023-10-30T12:47:11.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05ed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa060153c80 0x7fa06015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38308/0 (socket says 172.21.15.8:38308) 2023-10-30T12:47:11.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05ed9d640 1 -- 172.21.15.8:0/536697197 learned_addr learned my addr 172.21.15.8:0/536697197 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:11.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 --2- 172.21.15.8:0/536697197 >> v2:172.21.15.8:3300/0 conn(0x7fa0600c4a10 0x7fa06015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05f59e640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa0600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05e59c640 1 -- 172.21.15.8:0/536697197 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807917067 0 0) 0x7fa060159f20 con 0x7fa060153c80 2023-10-30T12:47:11.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05e59c640 1 -- 172.21.15.8:0/536697197 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa04c003530 con 0x7fa060153c80 2023-10-30T12:47:11.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 -- 172.21.15.8:0/536697197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 msgr2=0x7fa0600ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:11.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa0600ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 -- 172.21.15.8:0/536697197 >> v1:172.21.15.8:6789/0 conn(0x7fa060153c80 legacy=0x7fa06015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 -- 172.21.15.8:0/536697197 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06015d620 con 0x7fa0600c4a10 2023-10-30T12:47:11.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa057fff640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0600c4a10 0x7fa06015cf90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa0480097c0 tx=0x7fa04801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6224bf0d3020d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:11.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05e59c640 1 -- 172.21.15.8:0/536697197 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0480a8970 con 0x7fa0600c4a10 2023-10-30T12:47:11.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05e59c640 1 -- 172.21.15.8:0/536697197 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0480a8ad0 con 0x7fa0600c4a10 2023-10-30T12:47:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.527+0000 7fa05e59c640 1 -- 172.21.15.8:0/536697197 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0480a8e60 con 0x7fa0600c4a10 2023-10-30T12:47:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- 172.21.15.8:0/536697197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0600c4a10 msgr2=0x7fa06015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0600c4a10 0x7fa06015cf90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa0480097c0 tx=0x7fa04801ec70 comp rx=0 tx=0).stop 2023-10-30T12:47:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- 172.21.15.8:0/536697197 shutdown_connections 2023-10-30T12:47:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa0600ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- 172.21.15.8:0/536697197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0600c4a10 0x7fa06015cf90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- 172.21.15.8:0/536697197 >> 172.21.15.8:0/536697197 conn(0x7fa0600715c0 msgr2=0x7fa0600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- 172.21.15.8:0/536697197 shutdown_connections 2023-10-30T12:47:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- 172.21.15.8:0/536697197 wait complete. 2023-10-30T12:47:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 Processor -- start 2023-10-30T12:47:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- start start 2023-10-30T12:47:11.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa060155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 0x7fa060149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600c5d10 con 0x7fa060153c80 2023-10-30T12:47:11.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06014a4e0 con 0x7fa0600c4a10 2023-10-30T12:47:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa065017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600c6240 con 0x7fa0600c42a0 2023-10-30T12:47:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa05f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa05f59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37760/0 (socket says 172.21.15.8:37760) 2023-10-30T12:47:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa05f59e640 1 -- 172.21.15.8:0/2038274614 learned_addr learned my addr 172.21.15.8:0/2038274614 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa05ed9d640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa060155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.531+0000 7fa057fff640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 0x7fa060149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 -- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 msgr2=0x7fa060155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa060155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 -- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 msgr2=0x7fa060149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 0x7fa060149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 -- 172.21.15.8:0/2038274614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009420 con 0x7fa060153c80 2023-10-30T12:47:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa057fff640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 0x7fa060149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:11.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa05f59e640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa05800efc0 tx=0x7fa05800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058014480 con 0x7fa060153c80 2023-10-30T12:47:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05800f040 con 0x7fa060153c80 2023-10-30T12:47:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0601f2b40 con 0x7fa060153c80 2023-10-30T12:47:11.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058014820 con 0x7fa060153c80 2023-10-30T12:47:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601ee970 con 0x7fa060153c80 2023-10-30T12:47:11.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06004e170 con 0x7fa060153c80 2023-10-30T12:47:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa058014980 con 0x7fa060153c80 2023-10-30T12:47:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 0x7fa040068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:11.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.535+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fa058084a10 con 0x7fa060153c80 2023-10-30T12:47:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.539+0000 7fa05ed9d640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 0x7fa040068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.539+0000 7fa05ed9d640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 0x7fa040068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0500011f0 tx=0x7fa050007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:11.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.539+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa058051db0 con 0x7fa060153c80 2023-10-30T12:47:11.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0600cad40 con 0x7fa060153c80 2023-10-30T12:47:11.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa0577fe640 1 -- 172.21.15.8:0/2038274614 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7fa058018020 con 0x7fa060153c80 2023-10-30T12:47:11.737 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:11.737 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:11.738 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:11.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 msgr2=0x7fa040068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 0x7fa040068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0500011f0 tx=0x7fa050007d40 comp rx=0 tx=0).stop 2023-10-30T12:47:11.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 msgr2=0x7fa0601ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa05800efc0 tx=0x7fa05800c490 comp rx=0 tx=0).stop 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 shutdown_connections 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa0400664a0 0x7fa040068960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600c4a10 0x7fa060149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa060153c80 0x7fa0601ef170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 --2- 172.21.15.8:0/2038274614 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0600c42a0 0x7fa060155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:11.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.731+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 >> 172.21.15.8:0/2038274614 conn(0x7fa0600715c0 msgr2=0x7fa060159230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:11.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.735+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 shutdown_connections 2023-10-30T12:47:11.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:11.735+0000 7fa065017640 1 -- 172.21.15.8:0/2038274614 wait complete. 2023-10-30T12:47:11.758 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:13.759 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:13.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.907+0000 7f75ac3d3640 1 Processor -- start 2023-10-30T12:47:13.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.907+0000 7f75ac3d3640 1 -- start start 2023-10-30T12:47:13.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75a40c5ba0 0x7f75a40c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 0x7f75a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c9b40 0x7f75a406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a415d990 con 0x7f75a40c5ba0 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a4153f60 con 0x7f75a40c5f80 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75ac3d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a415cda0 con 0x7f75a40c9b40 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75a40c5ba0 0x7f75a40c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75a40c5ba0 0x7f75a40c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37772/0 (socket says 172.21.15.8:37772) 2023-10-30T12:47:13.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 -- 172.21.15.8:0/2337458511 learned_addr learned my addr 172.21.15.8:0/2337458511 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:13.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75a996e640 1 --2- 172.21.15.8:0/2337458511 >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 0x7f75a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa970640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c9b40 0x7f75a406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 -- 172.21.15.8:0/2337458511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c9b40 msgr2=0x7f75a406a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:13.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c9b40 0x7f75a406a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 -- 172.21.15.8:0/2337458511 >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 msgr2=0x7f75a4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:13.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 --2- 172.21.15.8:0/2337458511 >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 0x7f75a4069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75aa16f640 1 -- 172.21.15.8:0/2337458511 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a406f490 con 0x7f75a40c5ba0 2023-10-30T12:47:13.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.911+0000 7f75a996e640 1 --2- 172.21.15.8:0/2337458511 >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 0x7f75a4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:13.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75aa16f640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5ba0 0x7f75a40c70f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75a406e290 tx=0x7f75a001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a3d165d7853273 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:13.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75a916d640 1 -- 172.21.15.8:0/2337458511 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00ac410 con 0x7f75a40c5ba0 2023-10-30T12:47:13.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75a916d640 1 -- 172.21.15.8:0/2337458511 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a00ac570 con 0x7f75a40c5ba0 2023-10-30T12:47:13.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75a916d640 1 -- 172.21.15.8:0/2337458511 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00ac900 con 0x7f75a40c5ba0 2023-10-30T12:47:13.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- 172.21.15.8:0/2337458511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5ba0 msgr2=0x7f75a40c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:13.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5ba0 0x7f75a40c70f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75a406e290 tx=0x7f75a001c500 comp rx=0 tx=0).stop 2023-10-30T12:47:13.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- 172.21.15.8:0/2337458511 shutdown_connections 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c9b40 0x7f75a406a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/2337458511 >> v2:172.21.15.46:3300/0 conn(0x7f75a40c5f80 0x7f75a4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/2337458511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5ba0 0x7f75a40c70f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- 172.21.15.8:0/2337458511 >> 172.21.15.8:0/2337458511 conn(0x7f75a406afa0 msgr2=0x7f75a406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- 172.21.15.8:0/2337458511 shutdown_connections 2023-10-30T12:47:13.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- 172.21.15.8:0/2337458511 wait complete. 2023-10-30T12:47:13.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 Processor -- start 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 -- start start 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.915+0000 7f75ac3d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 0x7f75a4148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a40c9b40 0x7f75a40c2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4157fc0 con 0x7f75a40c5f80 2023-10-30T12:47:13.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a414b150 con 0x7f75a40c9b40 2023-10-30T12:47:13.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a414ae70 con 0x7f75a40c5ba0 2023-10-30T12:47:13.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37786/0 (socket says 172.21.15.8:37786) 2023-10-30T12:47:13.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 -- 172.21.15.8:0/1592706804 learned_addr learned my addr 172.21.15.8:0/1592706804 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:13.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75aa16f640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 0x7f75a4148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75aa970640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a40c9b40 0x7f75a40c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 -- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 msgr2=0x7f75a4148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:13.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 0x7f75a4148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 -- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a40c9b40 msgr2=0x7f75a40c2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a40c9b40 0x7f75a40c2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 -- 172.21.15.8:0/1592706804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00094d0 con 0x7f75a40c5f80 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75aa16f640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 0x7f75a4148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75a996e640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f759400d800 tx=0x7f759400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75940096a0 con 0x7f75a40c5f80 2023-10-30T12:47:13.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594009800 con 0x7f75a40c5f80 2023-10-30T12:47:13.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594009990 con 0x7f75a40c5f80 2023-10-30T12:47:13.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a4157690 con 0x7f75a40c5f80 2023-10-30T12:47:13.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a4157c50 con 0x7f75a40c5f80 2023-10-30T12:47:13.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a404e170 con 0x7f75a40c5f80 2023-10-30T12:47:13.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.919+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7594009af0 con 0x7f75a40c5f80 2023-10-30T12:47:13.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.923+0000 7f759affd640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 0x7f7588068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:13.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.923+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f75940842b0 con 0x7f75a40c5f80 2023-10-30T12:47:13.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.923+0000 7f75aa16f640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 0x7f7588068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:13.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.923+0000 7f75aa16f640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 0x7f7588068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75a00ae000 tx=0x7f75a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:13.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:13.927+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7594051660 con 0x7f75a40c5f80 2023-10-30T12:47:14.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.115+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75a40cbf00 con 0x7f75a40c5f80 2023-10-30T12:47:14.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f759affd640 1 -- 172.21.15.8:0/1592706804 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f7594054c80 con 0x7f75a40c5f80 2023-10-30T12:47:14.123 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:14.124 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:14.124 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:14.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 msgr2=0x7f7588068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:14.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 0x7f7588068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75a00ae000 tx=0x7f75a0008040 comp rx=0 tx=0).stop 2023-10-30T12:47:14.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 msgr2=0x7f75a414dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:14.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f759400d800 tx=0x7f759400dcc0 comp rx=0 tx=0).stop 2023-10-30T12:47:14.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 shutdown_connections 2023-10-30T12:47:14.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75880664a0 0x7f7588068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a40c9b40 0x7f75a40c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75a40c5f80 0x7f75a414dd70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 --2- 172.21.15.8:0/1592706804 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75a40c5ba0 0x7f75a4148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 >> 172.21.15.8:0/1592706804 conn(0x7f75a406afa0 msgr2=0x7f75a40cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 shutdown_connections 2023-10-30T12:47:14.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:14.119+0000 7f75ac3d3640 1 -- 172.21.15.8:0/1592706804 wait complete. 2023-10-30T12:47:14.146 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:14.146 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:14.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:16.147 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:16.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.295+0000 7f38fe73e640 1 Processor -- start 2023-10-30T12:47:16.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.295+0000 7f38fe73e640 1 -- start start 2023-10-30T12:47:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fe73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f8157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fe73e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38f806c420 0x7f38f8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fe73e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f8071e00 con 0x7f38f806b760 2023-10-30T12:47:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fe73e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f8070020 con 0x7f38f806c420 2023-10-30T12:47:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fe73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8070f10 con 0x7f38f806bb40 2023-10-30T12:47:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38f806c420 0x7f38f8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38f77fe640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f8157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38f806c420 0x7f38f8157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:53102/0 (socket says 172.21.15.8:53102) 2023-10-30T12:47:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38f7fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38f806b760 0x7f38f815e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:42026/0 (socket says 172.21.15.8:42026) 2023-10-30T12:47:16.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 -- 172.21.15.8:0/139425917 learned_addr learned my addr 172.21.15.8:0/139425917 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:16.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 -- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 msgr2=0x7f38f8157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:16.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 --2- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f8157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 -- 172.21.15.8:0/139425917 >> v1:172.21.15.46:6789/0 conn(0x7f38f806b760 legacy=0x7f38f815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.299+0000 7f38fccdb640 1 -- 172.21.15.8:0/139425917 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80711f0 con 0x7f38f806c420 2023-10-30T12:47:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fccdb640 1 --2- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806c420 0x7f38f8157f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f001c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15df738eaab9ec6d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38f6ffd640 1 -- 172.21.15.8:0/139425917 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00a58a0 con 0x7f38f806c420 2023-10-30T12:47:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38f6ffd640 1 -- 172.21.15.8:0/139425917 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f00a5a00 con 0x7f38f806c420 2023-10-30T12:47:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38f6ffd640 1 -- 172.21.15.8:0/139425917 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00a5d90 con 0x7f38f806c420 2023-10-30T12:47:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806c420 msgr2=0x7f38f8157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 --2- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806c420 0x7f38f8157f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f001c9e0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- 172.21.15.8:0/139425917 shutdown_connections 2023-10-30T12:47:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 --2- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806c420 0x7f38f8157f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 --2- 172.21.15.8:0/139425917 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f8157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- 172.21.15.8:0/139425917 >> 172.21.15.8:0/139425917 conn(0x7f38f80cac70 msgr2=0x7f38f80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- 172.21.15.8:0/139425917 shutdown_connections 2023-10-30T12:47:16.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- 172.21.15.8:0/139425917 wait complete. 2023-10-30T12:47:16.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 Processor -- start 2023-10-30T12:47:16.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.303+0000 7f38fe73e640 1 -- start start 2023-10-30T12:47:16.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 0x7f38f81f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f814be60 con 0x7f38f806c420 2023-10-30T12:47:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f814c120 con 0x7f38f806b760 2023-10-30T12:47:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f814c400 con 0x7f38f806bb40 2023-10-30T12:47:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37800/0 (socket says 172.21.15.8:37800) 2023-10-30T12:47:16.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 -- 172.21.15.8:0/2061395795 learned_addr learned my addr 172.21.15.8:0/2061395795 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:16.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f77fe640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f7fff640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 0x7f38f81f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 -- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 msgr2=0x7f38f80c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f80c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 -- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 msgr2=0x7f38f81f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 0x7f38f81f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 -- 172.21.15.8:0/2061395795 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0009b30 con 0x7f38f806c420 2023-10-30T12:47:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f77fe640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f7fff640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 0x7f38f81f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:16.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fccdb640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f38f001a610 tx=0x7f38f0019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:16.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00a5d70 con 0x7f38f806c420 2023-10-30T12:47:16.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f00b8470 con 0x7f38f806c420 2023-10-30T12:47:16.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f814c6e0 con 0x7f38f806c420 2023-10-30T12:47:16.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00b8820 con 0x7f38f806c420 2023-10-30T12:47:16.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f814cca0 con 0x7f38f806c420 2023-10-30T12:47:16.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f8066510 con 0x7f38f806c420 2023-10-30T12:47:16.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.307+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38f00b8980 con 0x7f38f806c420 2023-10-30T12:47:16.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.311+0000 7f38f4ff9640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 0x7f38e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:16.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.311+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f38f011f2e0 con 0x7f38f806c420 2023-10-30T12:47:16.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.311+0000 7f38f7fff640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 0x7f38e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:16.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.311+0000 7f38f7fff640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 0x7f38e4068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f70 tx=0x7f38e8007d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:16.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.315+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f38f00ec720 con 0x7f38f806c420 2023-10-30T12:47:16.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38f814ce90 con 0x7f38f806c420 2023-10-30T12:47:16.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38f4ff9640 1 -- 172.21.15.8:0/2061395795 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f38f00efd40 con 0x7f38f806c420 2023-10-30T12:47:16.509 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:16.509 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:16.511 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:16.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 msgr2=0x7f38e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 0x7f38e4068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f70 tx=0x7f38e8007d50 comp rx=0 tx=0).stop 2023-10-30T12:47:16.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 msgr2=0x7f38f814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:16.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f38f001a610 tx=0x7f38f0019780 comp rx=0 tx=0).stop 2023-10-30T12:47:16.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 shutdown_connections 2023-10-30T12:47:16.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f38e40664a0 0x7f38e4068960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38f806c420 0x7f38f814e640 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f38f806bb40 0x7f38f80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 --2- 172.21.15.8:0/2061395795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f806b760 0x7f38f81f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:16.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 >> 172.21.15.8:0/2061395795 conn(0x7f38f80cac70 msgr2=0x7f38f815e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:16.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 shutdown_connections 2023-10-30T12:47:16.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:16.503+0000 7f38fe73e640 1 -- 172.21.15.8:0/2061395795 wait complete. 2023-10-30T12:47:16.530 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:18.532 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:18.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.679+0000 7f1fbc09c640 1 Processor -- start 2023-10-30T12:47:18.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.679+0000 7f1fbc09c640 1 -- start start 2023-10-30T12:47:18.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fbc09c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb40701a0 0x7f1fb415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fbc09c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1fb406b2f0 0x7f1fb406b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fbc09c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40c2aa0 con 0x7f1fb41537f0 2023-10-30T12:47:18.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fbc09c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb40c3ab0 con 0x7f1fb406b2f0 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fbc09c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415a820 con 0x7f1fb40701a0 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fba639640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1fb406b2f0 0x7f1fb406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fb9e38640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f1fb41537f0 0x7f1fb40ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38312/0 (socket says 172.21.15.8:38312) 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fb9e38640 1 -- 172.21.15.8:0/2394585628 learned_addr learned my addr 172.21.15.8:0/2394585628 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fb9637640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb40701a0 0x7f1fb415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fba639640 1 -- 172.21.15.8:0/2394585628 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb40701a0 msgr2=0x7f1fb415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:18.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fba639640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb40701a0 0x7f1fb415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fba639640 1 -- 172.21.15.8:0/2394585628 >> v1:172.21.15.8:6789/0 conn(0x7f1fb41537f0 legacy=0x7f1fb40ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fba639640 1 -- 172.21.15.8:0/2394585628 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb406bca0 con 0x7f1fb406b2f0 2023-10-30T12:47:18.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.683+0000 7f1fb8e36640 1 -- 172.21.15.8:0/2394585628 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233611418 0 0) 0x7f1fb40c2aa0 con 0x7f1fb41537f0 2023-10-30T12:47:18.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fba639640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb406b2f0 0x7f1fb406b760 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba7d05dfe261908 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:18.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fb8e36640 1 -- 172.21.15.8:0/2394585628 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0af600 con 0x7f1fb406b2f0 2023-10-30T12:47:18.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fb8e36640 1 -- 172.21.15.8:0/2394585628 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac0ab050 con 0x7f1fb406b2f0 2023-10-30T12:47:18.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fb8e36640 1 -- 172.21.15.8:0/2394585628 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0af980 con 0x7f1fb406b2f0 2023-10-30T12:47:18.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2394585628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb406b2f0 msgr2=0x7f1fb406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb406b2f0 0x7f1fb406b760 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac01f130 comp rx=0 tx=0).stop 2023-10-30T12:47:18.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2394585628 shutdown_connections 2023-10-30T12:47:18.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb406b2f0 0x7f1fb406b760 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2394585628 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb40701a0 0x7f1fb415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2394585628 >> 172.21.15.8:0/2394585628 conn(0x7f1fb4071fb0 msgr2=0x7f1fb40723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:18.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2394585628 shutdown_connections 2023-10-30T12:47:18.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2394585628 wait complete. 2023-10-30T12:47:18.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 Processor -- start 2023-10-30T12:47:18.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.687+0000 7f1fbc09c640 1 -- start start 2023-10-30T12:47:18.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 0x7f1fb414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb41537f0 0x7f1fb41f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415b490 con 0x7f1fb40701a0 2023-10-30T12:47:18.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41efbd0 con 0x7f1fb406b2f0 2023-10-30T12:47:18.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41562e0 con 0x7f1fb41537f0 2023-10-30T12:47:18.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37824/0 (socket says 172.21.15.8:37824) 2023-10-30T12:47:18.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 -- 172.21.15.8:0/2530042015 learned_addr learned my addr 172.21.15.8:0/2530042015 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:18.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9e38640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 0x7f1fb414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fba639640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb41537f0 0x7f1fb41f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 -- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb41537f0 msgr2=0x7f1fb41f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb41537f0 0x7f1fb41f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 -- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 msgr2=0x7f1fb414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 0x7f1fb414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 -- 172.21.15.8:0/2530042015 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009a70 con 0x7f1fb40701a0 2023-10-30T12:47:18.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9e38640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 0x7f1fb414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fb9637640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400d830 tx=0x7f1fa400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400b6c0 con 0x7f1fb40701a0 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400b820 con 0x7f1fb40701a0 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb4156550 con 0x7f1fb40701a0 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400bb90 con 0x7f1fb40701a0 2023-10-30T12:47:18.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb41f0100 con 0x7f1fb40701a0 2023-10-30T12:47:18.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb404e170 con 0x7f1fb40701a0 2023-10-30T12:47:18.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.691+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1fa401e460 con 0x7f1fb40701a0 2023-10-30T12:47:18.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.695+0000 7f1faa7fc640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 0x7f1f98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:18.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.695+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f1fa40847f0 con 0x7f1fb40701a0 2023-10-30T12:47:18.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.695+0000 7f1fb9e38640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 0x7f1f98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:18.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.695+0000 7f1fb9e38640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 0x7f1f98068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007fd0 tx=0x7f1fb0007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:18.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.699+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1fa4051a90 con 0x7f1fb40701a0 2023-10-30T12:47:18.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.895+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fb4151e70 con 0x7f1fb40701a0 2023-10-30T12:47:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.895+0000 7f1faa7fc640 1 -- 172.21.15.8:0/2530042015 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f1fa40550b0 con 0x7f1fb40701a0 2023-10-30T12:47:18.903 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:18.903 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:18.904 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 msgr2=0x7f1f98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 0x7f1f98068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007fd0 tx=0x7f1fb0007c20 comp rx=0 tx=0).stop 2023-10-30T12:47:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 msgr2=0x7f1fb414cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400d830 tx=0x7f1fa400dcf0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 shutdown_connections 2023-10-30T12:47:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1f980664a0 0x7f1f98068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1fb41537f0 0x7f1fb41f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1fb40701a0 0x7f1fb414cf40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 --2- 172.21.15.8:0/2530042015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb406b2f0 0x7f1fb414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 >> 172.21.15.8:0/2530042015 conn(0x7f1fb4071fb0 msgr2=0x7f1fb41513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 shutdown_connections 2023-10-30T12:47:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:18.899+0000 7f1fbc09c640 1 -- 172.21.15.8:0/2530042015 wait complete. 2023-10-30T12:47:18.924 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:20.925 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:21.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 Processor -- start 2023-10-30T12:47:21.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 -- start start 2023-10-30T12:47:21.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49b80c4a10 0x7f49b815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f49b80c42a0 0x7f49b80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b8159f20 con 0x7f49b8153c80 2023-10-30T12:47:21.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b815dec0 con 0x7f49b80c42a0 2023-10-30T12:47:21.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bf8d6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b81599a0 con 0x7f49b80c4a10 2023-10-30T12:47:21.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f49b80c42a0 0x7f49b80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f49b80c42a0 0x7f49b80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33628/0 (socket says 172.21.15.8:33628) 2023-10-30T12:47:21.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 -- 172.21.15.8:0/103646177 learned_addr learned my addr 172.21.15.8:0/103646177 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:21.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bce71640 1 --2- 172.21.15.8:0/103646177 >> v2:172.21.15.46:3300/0 conn(0x7f49b80c4a10 0x7f49b815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 -- 172.21.15.8:0/103646177 >> v2:172.21.15.46:3300/0 conn(0x7f49b80c4a10 msgr2=0x7f49b815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:21.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 --2- 172.21.15.8:0/103646177 >> v2:172.21.15.46:3300/0 conn(0x7f49b80c4a10 0x7f49b815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 -- 172.21.15.8:0/103646177 >> v1:172.21.15.46:6789/0 conn(0x7f49b8153c80 legacy=0x7f49b815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.071+0000 7f49bde73640 1 -- 172.21.15.8:0/103646177 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b815d620 con 0x7f49b80c42a0 2023-10-30T12:47:21.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bde73640 1 --2- 172.21.15.8:0/103646177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b80c42a0 0x7f49b80ca1e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f49b806be50 tx=0x7f49b001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4728688c3f2f2c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:21.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49affff640 1 -- 172.21.15.8:0/103646177 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00af400 con 0x7f49b80c42a0 2023-10-30T12:47:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49affff640 1 -- 172.21.15.8:0/103646177 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b00ab050 con 0x7f49b80c42a0 2023-10-30T12:47:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49affff640 1 -- 172.21.15.8:0/103646177 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00af780 con 0x7f49b80c42a0 2023-10-30T12:47:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 -- 172.21.15.8:0/103646177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b80c42a0 msgr2=0x7f49b80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/103646177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b80c42a0 0x7f49b80ca1e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f49b806be50 tx=0x7f49b001f1c0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 -- 172.21.15.8:0/103646177 shutdown_connections 2023-10-30T12:47:21.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/103646177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b80c42a0 0x7f49b80ca1e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/103646177 >> v2:172.21.15.46:3300/0 conn(0x7f49b80c4a10 0x7f49b815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 -- 172.21.15.8:0/103646177 >> 172.21.15.8:0/103646177 conn(0x7f49b80715c0 msgr2=0x7f49b80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:21.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 -- 172.21.15.8:0/103646177 shutdown_connections 2023-10-30T12:47:21.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.075+0000 7f49bf8d6640 1 -- 172.21.15.8:0/103646177 wait complete. 2023-10-30T12:47:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 Processor -- start 2023-10-30T12:47:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- start start 2023-10-30T12:47:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 0x7f49b81f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 0x7f49b80ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81f2f80 con 0x7f49b8153c80 2023-10-30T12:47:21.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81f2ca0 con 0x7f49b80c4a10 2023-10-30T12:47:21.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80cfa30 con 0x7f49b80c42a0 2023-10-30T12:47:21.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33642/0 (socket says 172.21.15.8:33642) 2023-10-30T12:47:21.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 -- 172.21.15.8:0/1454128157 learned_addr learned my addr 172.21.15.8:0/1454128157 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:21.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bd672640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 0x7f49b81f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 -- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 msgr2=0x7f49b81f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 0x7f49b81f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 -- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 msgr2=0x7f49b80ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bce71640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 0x7f49b80ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 0x7f49b80ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 -- 172.21.15.8:0/1454128157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0009b30 con 0x7f49b8153c80 2023-10-30T12:47:21.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bd672640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 0x7f49b81f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:21.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bce71640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 0x7f49b80ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:47:21.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bde73640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f49b00b2040 tx=0x7f49b001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:21.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00afdc0 con 0x7f49b8153c80 2023-10-30T12:47:21.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0009510 con 0x7f49b8153c80 2023-10-30T12:47:21.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b80cfd10 con 0x7f49b8153c80 2023-10-30T12:47:21.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00b8640 con 0x7f49b8153c80 2023-10-30T12:47:21.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b80cdae0 con 0x7f49b8153c80 2023-10-30T12:47:21.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.079+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b804e170 con 0x7f49b8153c80 2023-10-30T12:47:21.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.083+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49b00b6040 con 0x7f49b8153c80 2023-10-30T12:47:21.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.083+0000 7f49adffb640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 0x7f499c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:21.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.083+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f49b0121fa0 con 0x7f49b8153c80 2023-10-30T12:47:21.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.083+0000 7f49bd672640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 0x7f499c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:21.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.083+0000 7f49bd672640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 0x7f499c068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f49b4000f70 tx=0x7f49b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:21.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.087+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49b00ef240 con 0x7f49b8153c80 2023-10-30T12:47:21.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.275+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49b80cad40 con 0x7f49b8153c80 2023-10-30T12:47:21.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.275+0000 7f49adffb640 1 -- 172.21.15.8:0/1454128157 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6507 (secure 0 0 0) 0x7f49b00b4070 con 0x7f49b8153c80 2023-10-30T12:47:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:21.283 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:45:35.211861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"c","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.8:6839/710307090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":710307090},{"type":"v1","addr":"172.21.15.8:6839","nonce":710307090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:21.284 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-10-30T12:47:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 msgr2=0x7f499c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 0x7f499c068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f49b4000f70 tx=0x7f49b4009040 comp rx=0 tx=0).stop 2023-10-30T12:47:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 msgr2=0x7f49b80ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:21.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f49b00b2040 tx=0x7f49b001bf60 comp rx=0 tx=0).stop 2023-10-30T12:47:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 shutdown_connections 2023-10-30T12:47:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f499c0664a0 0x7f499c068960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b80c4a10 0x7f49b80ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49b8153c80 0x7f49b80ce540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 --2- 172.21.15.8:0/1454128157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f49b80c42a0 0x7f49b81f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 >> 172.21.15.8:0/1454128157 conn(0x7f49b80715c0 msgr2=0x7f49b8151450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 shutdown_connections 2023-10-30T12:47:21.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:21.279+0000 7f49bf8d6640 1 -- 172.21.15.8:0/1454128157 wait complete. 2023-10-30T12:47:21.304 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:47:23.305 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:23.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.447+0000 7ff7293d3640 1 Processor -- start 2023-10-30T12:47:23.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.447+0000 7ff7293d3640 1 -- start start 2023-10-30T12:47:23.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff724153c80 0x7ff72415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7240c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff72415a3b0 0x7ff7240ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72415dec0 con 0x7ff724153c80 2023-10-30T12:47:23.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7241599a0 con 0x7ff72415a3b0 2023-10-30T12:47:23.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7293d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff724159f20 con 0x7ff7240c4a10 2023-10-30T12:47:23.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff724153c80 0x7ff72415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff724153c80 0x7ff72415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33658/0 (socket says 172.21.15.8:33658) 2023-10-30T12:47:23.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 -- 172.21.15.8:0/3264036058 learned_addr learned my addr 172.21.15.8:0/3264036058 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:23.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7237fe640 1 --2- 172.21.15.8:0/3264036058 >> v2:172.21.15.46:3300/0 conn(0x7ff72415a3b0 0x7ff7240ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff7227fc640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7240c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 -- 172.21.15.8:0/3264036058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 msgr2=0x7ff7240c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:23.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7240c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 -- 172.21.15.8:0/3264036058 >> v2:172.21.15.46:3300/0 conn(0x7ff72415a3b0 msgr2=0x7ff7240ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:23.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 --2- 172.21.15.8:0/3264036058 >> v2:172.21.15.46:3300/0 conn(0x7ff72415a3b0 0x7ff7240ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.451+0000 7ff722ffd640 1 -- 172.21.15.8:0/3264036058 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240c47e0 con 0x7ff724153c80 2023-10-30T12:47:23.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff722ffd640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff72415cf90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7240cbbc0 tx=0x7ff71401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b27380e0b071434 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff721ffb640 1 -- 172.21.15.8:0/3264036058 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140a89b0 con 0x7ff724153c80 2023-10-30T12:47:23.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff721ffb640 1 -- 172.21.15.8:0/3264036058 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7140a8b10 con 0x7ff724153c80 2023-10-30T12:47:23.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff721ffb640 1 -- 172.21.15.8:0/3264036058 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140a8ea0 con 0x7ff724153c80 2023-10-30T12:47:23.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- 172.21.15.8:0/3264036058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 msgr2=0x7ff72415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff72415cf90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7240cbbc0 tx=0x7ff71401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- 172.21.15.8:0/3264036058 shutdown_connections 2023-10-30T12:47:23.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 --2- 172.21.15.8:0/3264036058 >> v2:172.21.15.46:3300/0 conn(0x7ff72415a3b0 0x7ff7240ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7240c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 --2- 172.21.15.8:0/3264036058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff72415cf90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- 172.21.15.8:0/3264036058 >> 172.21.15.8:0/3264036058 conn(0x7ff7240715c0 msgr2=0x7ff7240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:23.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- 172.21.15.8:0/3264036058 shutdown_connections 2023-10-30T12:47:23.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- 172.21.15.8:0/3264036058 wait complete. 2023-10-30T12:47:23.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 Processor -- start 2023-10-30T12:47:23.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.455+0000 7ff7293d3640 1 -- start start 2023-10-30T12:47:23.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7241efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 0x7ff72414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241f00a0 con 0x7ff724153c80 2023-10-30T12:47:23.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c8640 con 0x7ff72415a3b0 2023-10-30T12:47:23.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c8e70 con 0x7ff7240c4a10 2023-10-30T12:47:23.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33660/0 (socket says 172.21.15.8:33660) 2023-10-30T12:47:23.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 -- 172.21.15.8:0/4283113612 learned_addr learned my addr 172.21.15.8:0/4283113612 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:23.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff722ffd640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7241efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7237fe640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 0x7ff72414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 -- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 msgr2=0x7ff7241efb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7241efb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 -- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 msgr2=0x7ff72414f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 0x7ff72414f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 -- 172.21.15.8:0/4283113612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140094d0 con 0x7ff724153c80 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7237fe640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 0x7ff72414f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7227fc640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff71800d7d0 tx=0x7ff71800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180096a0 con 0x7ff724153c80 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718009800 con 0x7ff724153c80 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240c9150 con 0x7ff724153c80 2023-10-30T12:47:23.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718009bb0 con 0x7ff724153c80 2023-10-30T12:47:23.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7240c7620 con 0x7ff724153c80 2023-10-30T12:47:23.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.459+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72404e170 con 0x7ff724153c80 2023-10-30T12:47:23.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.463+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff718009d10 con 0x7ff724153c80 2023-10-30T12:47:23.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.463+0000 7ff70b7fe640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 0x7ff710068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.463+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7ff7180843b0 con 0x7ff724153c80 2023-10-30T12:47:23.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.463+0000 7ff722ffd640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 0x7ff710068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.463+0000 7ff722ffd640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 0x7ff710068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7140b2000 tx=0x7ff714009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.467+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff718051740 con 0x7ff724153c80 2023-10-30T12:47:23.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.663+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff72414dd10 con 0x7ff724153c80 2023-10-30T12:47:23.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.663+0000 7ff70b7fe640 1 -- 172.21.15.8:0/4283113612 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5724 (secure 0 0 0) 0x7ff718054d60 con 0x7ff724153c80 2023-10-30T12:47:23.669 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:23.670 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:22.043686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:23.670 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 13 2023-10-30T12:47:23.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 msgr2=0x7ff710068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 0x7ff710068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7140b2000 tx=0x7ff714009040 comp rx=0 tx=0).stop 2023-10-30T12:47:23.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 msgr2=0x7ff7240c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff71800d7d0 tx=0x7ff71800dc90 comp rx=0 tx=0).stop 2023-10-30T12:47:23.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 shutdown_connections 2023-10-30T12:47:23.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff7100664a0 0x7ff710068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff72415a3b0 0x7ff72414f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724153c80 0x7ff7240c8100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 --2- 172.21.15.8:0/4283113612 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff7240c4a10 0x7ff7241efb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 >> 172.21.15.8:0/4283113612 conn(0x7ff7240715c0 msgr2=0x7ff7240c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:23.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 shutdown_connections 2023-10-30T12:47:23.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.667+0000 7ff7293d3640 1 -- 172.21.15.8:0/4283113612 wait complete. 2023-10-30T12:47:23.692 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-30T12:47:23.692 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:47:23.693 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:23.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 Processor -- start 2023-10-30T12:47:23.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 -- start start 2023-10-30T12:47:23.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f10b4153c80 0x7f10b415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10b40c4a10 0x7f10b40c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b4159f20 con 0x7f10b415a3b0 2023-10-30T12:47:23.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b415dec0 con 0x7f10b4153c80 2023-10-30T12:47:23.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b930f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b41599a0 con 0x7f10b40c4a10 2023-10-30T12:47:23.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f10b4153c80 0x7f10b415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f10b4153c80 0x7f10b415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33670/0 (socket says 172.21.15.8:33670) 2023-10-30T12:47:23.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 -- 172.21.15.8:0/34435796 learned_addr learned my addr 172.21.15.8:0/34435796 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:23.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b27fc640 1 --2- 172.21.15.8:0/34435796 >> v2:172.21.15.46:3300/0 conn(0x7f10b40c4a10 0x7f10b40c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 -- 172.21.15.8:0/34435796 >> v2:172.21.15.46:3300/0 conn(0x7f10b40c4a10 msgr2=0x7f10b40c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:23.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 --2- 172.21.15.8:0/34435796 >> v2:172.21.15.46:3300/0 conn(0x7f10b40c4a10 0x7f10b40c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 -- 172.21.15.8:0/34435796 >> v1:172.21.15.46:6789/0 conn(0x7f10b415a3b0 legacy=0x7f10b40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:23.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.831+0000 7f10b2ffd640 1 -- 172.21.15.8:0/34435796 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b40c47e0 con 0x7f10b4153c80 2023-10-30T12:47:23.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b2ffd640 1 --2- 172.21.15.8:0/34435796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415cf90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10b406be50 tx=0x7f10a401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a80c3166d0f419e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b1ffb640 1 -- 172.21.15.8:0/34435796 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40a89b0 con 0x7f10b4153c80 2023-10-30T12:47:23.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b1ffb640 1 -- 172.21.15.8:0/34435796 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a40a8b10 con 0x7f10b4153c80 2023-10-30T12:47:23.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b1ffb640 1 -- 172.21.15.8:0/34435796 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40a8ea0 con 0x7f10b4153c80 2023-10-30T12:47:23.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 -- 172.21.15.8:0/34435796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 msgr2=0x7f10b415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 --2- 172.21.15.8:0/34435796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415cf90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10b406be50 tx=0x7f10a401ecb0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 -- 172.21.15.8:0/34435796 shutdown_connections 2023-10-30T12:47:23.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 --2- 172.21.15.8:0/34435796 >> v2:172.21.15.46:3300/0 conn(0x7f10b40c4a10 0x7f10b40c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 --2- 172.21.15.8:0/34435796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415cf90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 -- 172.21.15.8:0/34435796 >> 172.21.15.8:0/34435796 conn(0x7f10b40715c0 msgr2=0x7f10b40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:23.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 -- 172.21.15.8:0/34435796 shutdown_connections 2023-10-30T12:47:23.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.835+0000 7f10b930f640 1 -- 172.21.15.8:0/34435796 wait complete. 2023-10-30T12:47:23.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 Processor -- start 2023-10-30T12:47:23.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- start start 2023-10-30T12:47:23.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f10b40c4a10 0x7f10b41efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b415a3b0 0x7f10b415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b414e4c0 con 0x7f10b4153c80 2023-10-30T12:47:23.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41f0180 con 0x7f10b415a3b0 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b4152ea0 con 0x7f10b40c4a10 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33674/0 (socket says 172.21.15.8:33674) 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 -- 172.21.15.8:0/2095980806 learned_addr learned my addr 172.21.15.8:0/2095980806 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b37fe640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b415a3b0 0x7f10b415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b2ffd640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f10b40c4a10 0x7f10b41efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 -- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f10b40c4a10 msgr2=0x7f10b41efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f10b40c4a10 0x7f10b41efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 -- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b415a3b0 msgr2=0x7f10b415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:23.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b415a3b0 0x7f10b415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:23.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 -- 172.21.15.8:0/2095980806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40094d0 con 0x7f10b4153c80 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b27fc640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10a800d800 tx=0x7f10a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800b660 con 0x7f10b4153c80 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a800b7c0 con 0x7f10b4153c80 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800bb70 con 0x7f10b4153c80 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b415b6f0 con 0x7f10b4153c80 2023-10-30T12:47:23.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b4152570 con 0x7f10b4153c80 2023-10-30T12:47:23.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.839+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b4153050 con 0x7f10b4153c80 2023-10-30T12:47:23.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.843+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10a800bcd0 con 0x7f10b4153c80 2023-10-30T12:47:23.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.843+0000 7f109b7fe640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 0x7f10a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:23.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.843+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f10a8084390 con 0x7f10b4153c80 2023-10-30T12:47:23.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.843+0000 7f10b2ffd640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 0x7f10a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:23.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.843+0000 7f10b2ffd640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 0x7f10a0068990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10a40b2000 tx=0x7f10a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:23.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:23.847+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10b4153050 con 0x7f10b4153c80 2023-10-30T12:47:24.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10b4153010 con 0x7f10b4153c80 2023-10-30T12:47:24.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f109b7fe640 1 -- 172.21.15.8:0/2095980806 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f10a80551b0 con 0x7f10b4153c80 2023-10-30T12:47:24.041 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:24.041 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:22.043686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:24.042 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 13 2023-10-30T12:47:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 msgr2=0x7f10a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 0x7f10a0068990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10a40b2000 tx=0x7f10a4008040 comp rx=0 tx=0).stop 2023-10-30T12:47:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 msgr2=0x7f10b415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.035+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10a800d800 tx=0x7f10a800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:47:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 shutdown_connections 2023-10-30T12:47:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10a00664d0 0x7f10a0068990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b415a3b0 0x7f10b415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b4153c80 0x7f10b415ab60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 --2- 172.21.15.8:0/2095980806 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f10b40c4a10 0x7f10b41efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 >> 172.21.15.8:0/2095980806 conn(0x7f10b40715c0 msgr2=0x7f10b406bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 shutdown_connections 2023-10-30T12:47:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:24.039+0000 7f10b930f640 1 -- 172.21.15.8:0/2095980806 wait complete. 2023-10-30T12:47:24.063 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 13, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 13, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4244": { "addr": "172.21.15.8:6838/2992596118", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 2992596118, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 2992596118, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 13, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4276": { "addr": "172.21.15.46:6833/3888673612", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 3888673612, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 3888673612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4276, "incarnation": 9, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:47:22.043686+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 4276, "mds_2": 4244 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:47:26.064 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:26.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 Processor -- start 2023-10-30T12:47:26.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 -- start start 2023-10-30T12:47:26.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d4015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d40159f20 con 0x7f8d400c42a0 2023-10-30T12:47:26.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4015d620 con 0x7f8d400c4a10 2023-10-30T12:47:26.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d44d9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4015dec0 con 0x7f8d40153c80 2023-10-30T12:47:26.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8d400c42a0 0x7f8d400ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50210/0 (socket says 172.21.15.8:50210) 2023-10-30T12:47:26.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/291246363 learned_addr learned my addr 172.21.15.8:0/291246363 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:26.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3e59c640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d4015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337672474 0 0) 0x7f8d40159f20 con 0x7f8d400c42a0 2023-10-30T12:47:26.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d28003530 con 0x7f8d400c42a0 2023-10-30T12:47:26.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4120999424 0 0) 0x7f8d28003530 con 0x7f8d400c42a0 2023-10-30T12:47:26.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 msgr2=0x7f8d4015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:26.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d4015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 >> v1:172.21.15.46:6789/0 conn(0x7f8d400c4a10 legacy=0x7f8d4015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4015d900 con 0x7f8d400c42a0 2023-10-30T12:47:26.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f8d38001a00 con 0x7f8d400c42a0 2023-10-30T12:47:26.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 >> v1:172.21.15.8:6789/0 conn(0x7f8d400c42a0 legacy=0x7f8d400ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28003f30 0x7f8d280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d28004d10 0x7f8d28005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4015dec0 con 0x7f8d28004d10 2023-10-30T12:47:26.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2800bb00 con 0x7f8d28003f30 2023-10-30T12:47:26.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2800bcc0 con 0x7f8d40153c80 2023-10-30T12:47:26.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d28004d10 0x7f8d28005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3e59c640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3dd9b640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28003f30 0x7f8d280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 msgr2=0x7f8d280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d40153c80 0x7f8d280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28003f30 msgr2=0x7f8d280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:26.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28003f30 0x7f8d280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.215+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/291246363 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d28003530 con 0x7f8d28004d10 2023-10-30T12:47:26.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d28004d10 0x7f8d28005190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800a830 tx=0x7f8d3801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f434366acef4488f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:26.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380072d0 con 0x7f8d28004d10 2023-10-30T12:47:26.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d380a8070 con 0x7f8d28004d10 2023-10-30T12:47:26.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d3d59a640 1 -- 172.21.15.8:0/291246363 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380076f0 con 0x7f8d28004d10 2023-10-30T12:47:26.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 -- 172.21.15.8:0/291246363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d28004d10 msgr2=0x7f8d28005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/291246363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d28004d10 0x7f8d28005190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800a830 tx=0x7f8d3801d340 comp rx=0 tx=0).stop 2023-10-30T12:47:26.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/291246363 reap_dead start 2023-10-30T12:47:26.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 -- 172.21.15.8:0/291246363 shutdown_connections 2023-10-30T12:47:26.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 -- 172.21.15.8:0/291246363 >> 172.21.15.8:0/291246363 conn(0x7f8d400715c0 msgr2=0x7f8d400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:26.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 -- 172.21.15.8:0/291246363 shutdown_connections 2023-10-30T12:47:26.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.219+0000 7f8d44d9d640 1 -- 172.21.15.8:0/291246363 wait complete. 2023-10-30T12:47:26.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 Processor -- start 2023-10-30T12:47:26.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- start start 2023-10-30T12:47:26.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28004d10 0x7f8d4015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d401481f0 0x7f8d40156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401498a0 con 0x7f8d40156cd0 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40150260 con 0x7f8d28004d10 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4014ff80 con 0x7f8d401481f0 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33702/0 (socket says 172.21.15.8:33702) 2023-10-30T12:47:26.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/2474542627 learned_addr learned my addr 172.21.15.8:0/2474542627 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:26.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3dd9b640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d401481f0 0x7f8d40156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3e59c640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28004d10 0x7f8d4015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d401481f0 msgr2=0x7f8d40156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d401481f0 0x7f8d40156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28004d10 msgr2=0x7f8d4015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28004d10 0x7f8d4015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 -- 172.21.15.8:0/2474542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4015ba40 con 0x7f8d40156cd0 2023-10-30T12:47:26.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d3ed9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d3801d170 tx=0x7f8d28005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:26.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380011d0 con 0x7f8d40156cd0 2023-10-30T12:47:26.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d38004b70 con 0x7f8d40156cd0 2023-10-30T12:47:26.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38001800 con 0x7f8d40156cd0 2023-10-30T12:47:26.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4015bd20 con 0x7f8d40156cd0 2023-10-30T12:47:26.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.223+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4015c2e0 con 0x7f8d40156cd0 2023-10-30T12:47:26.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40153030 con 0x7f8d40156cd0 2023-10-30T12:47:26.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d380aa030 con 0x7f8d40156cd0 2023-10-30T12:47:26.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d2effd640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 0x7f8d200689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:26.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f8d3811d9d0 con 0x7f8d40156cd0 2023-10-30T12:47:26.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d3e59c640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 0x7f8d200689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:26.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.227+0000 7f8d3e59c640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 0x7f8d200689b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009b20 tx=0x7f8d30008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:26.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.231+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d380eadd0 con 0x7f8d40156cd0 2023-10-30T12:47:26.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.419+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d400c9c90 con 0x7f8d40156cd0 2023-10-30T12:47:26.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d2effd640 1 -- 172.21.15.8:0/2474542627 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f8d380ee3f0 con 0x7f8d40156cd0 2023-10-30T12:47:26.427 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:26.428 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:22.043686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:26.428 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 13 2023-10-30T12:47:26.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 msgr2=0x7f8d200689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 0x7f8d200689b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009b20 tx=0x7f8d30008040 comp rx=0 tx=0).stop 2023-10-30T12:47:26.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 msgr2=0x7f8d401f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d3801d170 tx=0x7f8d28005800 comp rx=0 tx=0).stop 2023-10-30T12:47:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 shutdown_connections 2023-10-30T12:47:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8d200664f0 0x7f8d200689b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d40156cd0 0x7f8d401f2920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8d401481f0 0x7f8d40156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 --2- 172.21.15.8:0/2474542627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d28004d10 0x7f8d4015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:26.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 >> 172.21.15.8:0/2474542627 conn(0x7f8d400715c0 msgr2=0x7f8d40150bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:26.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 shutdown_connections 2023-10-30T12:47:26.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:26.423+0000 7f8d44d9d640 1 -- 172.21.15.8:0/2474542627 wait complete. 2023-10-30T12:47:26.448 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:28.449 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:28.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.595+0000 7f0dd2809640 1 Processor -- start 2023-10-30T12:47:28.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 -- start start 2023-10-30T12:47:28.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dcc06dfc0 0x7f0dcc06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0dcc0c5460 0x7f0dcc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc070be0 con 0x7f0dcc15aa60 2023-10-30T12:47:28.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc06fc30 con 0x7f0dcc0c5460 2023-10-30T12:47:28.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd2809640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc15b360 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0dcc0c5460 0x7f0dcc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0dcc0c5460 0x7f0dcc0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33710/0 (socket says 172.21.15.8:33710) 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 -- 172.21.15.8:0/1966444870 learned_addr learned my addr 172.21.15.8:0/1966444870 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1807640 1 --2- 172.21.15.8:0/1966444870 >> v2:172.21.15.46:3300/0 conn(0x7f0dcc06dfc0 0x7f0dcc06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 -- 172.21.15.8:0/1966444870 >> v2:172.21.15.46:3300/0 conn(0x7f0dcc06dfc0 msgr2=0x7f0dcc06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:28.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 --2- 172.21.15.8:0/1966444870 >> v2:172.21.15.46:3300/0 conn(0x7f0dcc06dfc0 0x7f0dcc06d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 -- 172.21.15.8:0/1966444870 >> v1:172.21.15.46:6789/0 conn(0x7f0dcc15aa60 legacy=0x7f0dcc0c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:28.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.599+0000 7f0dd1006640 1 -- 172.21.15.8:0/1966444870 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc15b620 con 0x7f0dcc0c5460 2023-10-30T12:47:28.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd1006640 1 --2- 172.21.15.8:0/1966444870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc0706a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc074280 tx=0x7f0dbc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb44d298fc57b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd0805640 1 -- 172.21.15.8:0/1966444870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0aa5f0 con 0x7f0dcc0c5460 2023-10-30T12:47:28.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd0805640 1 -- 172.21.15.8:0/1966444870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0aa750 con 0x7f0dcc0c5460 2023-10-30T12:47:28.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd0805640 1 -- 172.21.15.8:0/1966444870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0aaae0 con 0x7f0dcc0c5460 2023-10-30T12:47:28.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- 172.21.15.8:0/1966444870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc0c5460 msgr2=0x7f0dcc0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 --2- 172.21.15.8:0/1966444870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc0706a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc074280 tx=0x7f0dbc01c490 comp rx=0 tx=0).stop 2023-10-30T12:47:28.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- 172.21.15.8:0/1966444870 shutdown_connections 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 --2- 172.21.15.8:0/1966444870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc0706a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 --2- 172.21.15.8:0/1966444870 >> v2:172.21.15.46:3300/0 conn(0x7f0dcc06dfc0 0x7f0dcc06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- 172.21.15.8:0/1966444870 >> 172.21.15.8:0/1966444870 conn(0x7f0dcc0caae0 msgr2=0x7f0dcc0caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:28.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- 172.21.15.8:0/1966444870 shutdown_connections 2023-10-30T12:47:28.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- 172.21.15.8:0/1966444870 wait complete. 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 Processor -- start 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.603+0000 7f0dd2809640 1 -- start start 2023-10-30T12:47:28.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0dcc15aa60 0x7f0dcc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc14dde0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc14ef80 con 0x7f0dcc0c5460 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0c21f0 con 0x7f0dcc15aa60 2023-10-30T12:47:28.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33718/0 (socket says 172.21.15.8:33718) 2023-10-30T12:47:28.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 -- 172.21.15.8:0/89095443 learned_addr learned my addr 172.21.15.8:0/89095443 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:28.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1006640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2008640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0dcc15aa60 0x7f0dcc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 -- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0dcc15aa60 msgr2=0x7f0dcc14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0dcc15aa60 0x7f0dcc14e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 -- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0c5460 msgr2=0x7f0dcc147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc147f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 -- 172.21.15.8:0/89095443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0094d0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd1807640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800d800 tx=0x7f0dc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800b6c0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800b820 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800bb90 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc1f2240 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc14db00 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.607+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc1f20a0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.611+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0dc800bcf0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.611+0000 7f0dc27fc640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 0x7f0db0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:28.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.611+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f0dc80843e0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.611+0000 7f0dd1006640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 0x7f0db0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:28.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.611+0000 7f0dd1006640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 0x7f0db0068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ae000 tx=0x7f0dbc0b5040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:28.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.615+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0dcc1f20a0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.811+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dcc1f20a0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.811+0000 7f0dc27fc640 1 -- 172.21.15.8:0/89095443 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5726 (secure 0 0 0) 0x7f0dcc1f20a0 con 0x7f0dcc06dfc0 2023-10-30T12:47:28.817 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:27.063295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:resolve","state_seq":35,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:28.818 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 14 2023-10-30T12:47:28.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 msgr2=0x7f0db0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 0x7f0db0068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ae000 tx=0x7f0dbc0b5040 comp rx=0 tx=0).stop 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 msgr2=0x7f0dcc14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800d800 tx=0x7f0dc800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 shutdown_connections 2023-10-30T12:47:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0db00664d0 0x7f0db0068990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dcc0c5460 0x7f0dcc147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0dcc15aa60 0x7f0dcc14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 --2- 172.21.15.8:0/89095443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0dcc06dfc0 0x7f0dcc14fe20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 >> 172.21.15.8:0/89095443 conn(0x7f0dcc0caae0 msgr2=0x7f0dcc0ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 shutdown_connections 2023-10-30T12:47:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:28.815+0000 7f0dd2809640 1 -- 172.21.15.8:0/89095443 wait complete. 2023-10-30T12:47:28.839 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:30.840 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:30.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 Processor -- start 2023-10-30T12:47:30.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 -- start start 2023-10-30T12:47:30.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25300c62b0 0x7f25301597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:30.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300cb180 0x7f253006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:30.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253015a770 con 0x7f2530153b50 2023-10-30T12:47:30.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2530159cf0 con 0x7f25300c62b0 2023-10-30T12:47:30.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f253603a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253006aa30 con 0x7f25300cb180 2023-10-30T12:47:30.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25300c62b0 0x7f25301597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:30.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f25300c62b0 0x7f25301597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47332/0 (socket says 172.21.15.8:47332) 2023-10-30T12:47:30.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 -- 172.21.15.8:0/2743217819 learned_addr learned my addr 172.21.15.8:0/2743217819 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:30.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2535839640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300cb180 0x7f253006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:30.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 -- 172.21.15.8:0/2743217819 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300cb180 msgr2=0x7f253006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:30.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300cb180 0x7f253006ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:30.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 -- 172.21.15.8:0/2743217819 >> v1:172.21.15.46:6789/0 conn(0x7f2530153b50 legacy=0x7f25300ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:30.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.991+0000 7f2534837640 1 -- 172.21.15.8:0/2743217819 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300c9d40 con 0x7f25300c62b0 2023-10-30T12:47:31.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f2534837640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25300c62b0 0x7f25301597b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2520009870 tx=0x7f252001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128eb04fe1783bd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f2527fff640 1 -- 172.21.15.8:0/2743217819 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200a8a30 con 0x7f25300c62b0 2023-10-30T12:47:31.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f2527fff640 1 -- 172.21.15.8:0/2743217819 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25200a8b90 con 0x7f25300c62b0 2023-10-30T12:47:31.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f2527fff640 1 -- 172.21.15.8:0/2743217819 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200b0440 con 0x7f25300c62b0 2023-10-30T12:47:31.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- 172.21.15.8:0/2743217819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25300c62b0 msgr2=0x7f25301597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25300c62b0 0x7f25301597b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2520009870 tx=0x7f252001ed30 comp rx=0 tx=0).stop 2023-10-30T12:47:31.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- 172.21.15.8:0/2743217819 shutdown_connections 2023-10-30T12:47:31.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300cb180 0x7f253006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 --2- 172.21.15.8:0/2743217819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25300c62b0 0x7f25301597b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- 172.21.15.8:0/2743217819 >> 172.21.15.8:0/2743217819 conn(0x7f2530065a80 msgr2=0x7f2530065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:31.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- 172.21.15.8:0/2743217819 shutdown_connections 2023-10-30T12:47:31.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- 172.21.15.8:0/2743217819 wait complete. 2023-10-30T12:47:31.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 Processor -- start 2023-10-30T12:47:31.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.995+0000 7f253603a640 1 -- start start 2023-10-30T12:47:31.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300c62b0 0x7f25301ef090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25300cb180 0x7f2530159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301ef5d0 con 0x7f2530153b50 2023-10-30T12:47:31.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2530159a60 con 0x7f25300cb180 2023-10-30T12:47:31.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25301f2a20 con 0x7f25300c62b0 2023-10-30T12:47:31.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47346/0 (socket says 172.21.15.8:47346) 2023-10-30T12:47:31.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 -- 172.21.15.8:0/3671843523 learned_addr learned my addr 172.21.15.8:0/3671843523 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:31.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2534837640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25300cb180 0x7f2530159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535038640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300c62b0 0x7f25301ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 -- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300c62b0 msgr2=0x7f25301ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300c62b0 0x7f25301ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 -- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25300cb180 msgr2=0x7f2530159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25300cb180 0x7f2530159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 -- 172.21.15.8:0/3671843523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200094d0 con 0x7f2530153b50 2023-10-30T12:47:31.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2535839640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2528007e70 tx=0x7f252800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528014620 con 0x7f2530153b50 2023-10-30T12:47:31.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25280147a0 con 0x7f2530153b50 2023-10-30T12:47:31.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528014b50 con 0x7f2530153b50 2023-10-30T12:47:31.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25301f1bc0 con 0x7f2530153b50 2023-10-30T12:47:31.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25301f20f0 con 0x7f2530153b50 2023-10-30T12:47:31.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:30.999+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2530067540 con 0x7f2530153b50 2023-10-30T12:47:31.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.003+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2528014cb0 con 0x7f2530153b50 2023-10-30T12:47:31.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.003+0000 7f2525ffb640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 0x7f2514068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:31.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.003+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f25280848c0 con 0x7f2530153b50 2023-10-30T12:47:31.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.003+0000 7f2535038640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 0x7f2514068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:31.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.003+0000 7f2535038640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 0x7f2514068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f252c007fd0 tx=0x7f252c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:31.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.007+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2528051bd0 con 0x7f2530153b50 2023-10-30T12:47:31.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25300ca980 con 0x7f2530153b50 2023-10-30T12:47:31.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f2525ffb640 1 -- 172.21.15.8:0/3671843523 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5726 (secure 0 0 0) 0x7f25280551f0 con 0x7f2530153b50 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:27.063295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:resolve","state_seq":35,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:31.204 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 14 2023-10-30T12:47:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 msgr2=0x7f2514068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 0x7f2514068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f252c007fd0 tx=0x7f252c009040 comp rx=0 tx=0).stop 2023-10-30T12:47:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 msgr2=0x7f2530150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2528007e70 tx=0x7f252800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 shutdown_connections 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f25140664a0 0x7f2514068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25300cb180 0x7f2530159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2530153b50 0x7f2530150b10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 --2- 172.21.15.8:0/3671843523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f25300c62b0 0x7f25301ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 >> 172.21.15.8:0/3671843523 conn(0x7f2530065a80 msgr2=0x7f25300c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 shutdown_connections 2023-10-30T12:47:31.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:31.199+0000 7f253603a640 1 -- 172.21.15.8:0/3671843523 wait complete. 2023-10-30T12:47:31.226 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:33.227 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:33.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.375+0000 7fed0c565640 1 Processor -- start 2023-10-30T12:47:33.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.375+0000 7fed0c565640 1 -- start start 2023-10-30T12:47:33.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0c565640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed040cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0c565640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed040c2500 0x7fed040c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0c565640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed04151aa0 con 0x7fed041516c0 2023-10-30T12:47:33.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0c565640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed040c74a0 con 0x7fed040c2500 2023-10-30T12:47:33.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0c565640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040c2e90 con 0x7fed040ccf60 2023-10-30T12:47:33.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed040c2500 0x7fed040c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed040c2500 0x7fed040c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47354/0 (socket says 172.21.15.8:47354) 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 -- 172.21.15.8:0/1577758491 learned_addr learned my addr 172.21.15.8:0/1577758491 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed09b00640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed040cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 -- 172.21.15.8:0/1577758491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 msgr2=0x7fed040cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed040cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 -- 172.21.15.8:0/1577758491 >> v1:172.21.15.46:6789/0 conn(0x7fed041516c0 legacy=0x7fed040c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 -- 172.21.15.8:0/1577758491 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04158af0 con 0x7fed040c2500 2023-10-30T12:47:33.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.379+0000 7fed0ab02640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed040c2950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e50 tx=0x7fecfc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db23222252b3a1f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed092ff640 1 -- 172.21.15.8:0/1577758491 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0a56e0 con 0x7fed040c2500 2023-10-30T12:47:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed092ff640 1 -- 172.21.15.8:0/1577758491 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc0a5840 con 0x7fed040c2500 2023-10-30T12:47:33.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed092ff640 1 -- 172.21.15.8:0/1577758491 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0a5bd0 con 0x7fed040c2500 2023-10-30T12:47:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- 172.21.15.8:0/1577758491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 msgr2=0x7fed040c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed040c2950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e50 tx=0x7fecfc01ca10 comp rx=0 tx=0).stop 2023-10-30T12:47:33.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- 172.21.15.8:0/1577758491 shutdown_connections 2023-10-30T12:47:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed040c2950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- 172.21.15.8:0/1577758491 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed040cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- 172.21.15.8:0/1577758491 >> 172.21.15.8:0/1577758491 conn(0x7fed04071fb0 msgr2=0x7fed040723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- 172.21.15.8:0/1577758491 shutdown_connections 2023-10-30T12:47:33.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- 172.21.15.8:0/1577758491 wait complete. 2023-10-30T12:47:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 Processor -- start 2023-10-30T12:47:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- start start 2023-10-30T12:47:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed041f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed041516c0 0x7fed041f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0414f750 con 0x7fed040c2500 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0414f9a0 con 0x7fed041516c0 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0c565640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0414fc80 con 0x7fed040ccf60 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.383+0000 7fed0a301640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47368/0 (socket says 172.21.15.8:47368) 2023-10-30T12:47:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 -- 172.21.15.8:0/3444401850 learned_addr learned my addr 172.21.15.8:0/3444401850 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed09b00640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed041f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0ab02640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed041516c0 0x7fed041f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 -- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 msgr2=0x7fed041f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed041f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 -- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed041516c0 msgr2=0x7fed041f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed041516c0 0x7fed041f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 -- 172.21.15.8:0/3444401850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009b30 con 0x7fed040c2500 2023-10-30T12:47:33.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0a301640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed0000c500 tx=0x7fed0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00005220 con 0x7fed040c2500 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00005380 con 0x7fed040c2500 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00007270 con 0x7fed040c2500 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0414ff60 con 0x7fed040c2500 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04150520 con 0x7fed040c2500 2023-10-30T12:47:33.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04073b80 con 0x7fed040c2500 2023-10-30T12:47:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed000073d0 con 0x7fed040c2500 2023-10-30T12:47:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.387+0000 7fecfaffd640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 0x7fece8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.391+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fed00083940 con 0x7fed040c2500 2023-10-30T12:47:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.391+0000 7fed09b00640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 0x7fece8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:33.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.391+0000 7fed09b00640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 0x7fece8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed04072500 tx=0x7fecf4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:33.399 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.395+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed00050c40 con 0x7fed040c2500 2023-10-30T12:47:33.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed0404e170 con 0x7fed040c2500 2023-10-30T12:47:33.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fecfaffd640 1 -- 172.21.15.8:0/3444401850 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5728 (secure 0 0 0) 0x7fed00054260 con 0x7fed040c2500 2023-10-30T12:47:33.589 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:33.589 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:33.054806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:33.589 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 15 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 msgr2=0x7fece8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 0x7fece8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed04072500 tx=0x7fecf4008040 comp rx=0 tx=0).stop 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 msgr2=0x7fed0415b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed0000c500 tx=0x7fed0000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 shutdown_connections 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fece80664a0 0x7fece8068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed041516c0 0x7fed041f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed040ccf60 0x7fed041f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 --2- 172.21.15.8:0/3444401850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed040c2500 0x7fed0415b0d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 >> 172.21.15.8:0/3444401850 conn(0x7fed04071fb0 msgr2=0x7fed041617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.583+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 shutdown_connections 2023-10-30T12:47:33.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:33.587+0000 7fed0c565640 1 -- 172.21.15.8:0/3444401850 wait complete. 2023-10-30T12:47:33.611 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:35.612 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:35.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 Processor -- start 2023-10-30T12:47:35.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 -- start start 2023-10-30T12:47:35.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3cd80c5f80 0x7f3cd80c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3cd80c9b40 0x7f3cd81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd815cda0 con 0x7f3cd80c5ba0 2023-10-30T12:47:35.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd815d990 con 0x7f3cd80c9b40 2023-10-30T12:47:35.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3ce046b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd8153f60 con 0x7f3cd80c5f80 2023-10-30T12:47:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3cdea08640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3cd80c9b40 0x7f3cd81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3cde207640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3cd80c5ba0 0x7f3cd8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40350/0 (socket says 172.21.15.8:40350) 2023-10-30T12:47:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3cde207640 1 -- 172.21.15.8:0/3580053104 learned_addr learned my addr 172.21.15.8:0/3580053104 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.759+0000 7f3cdda06640 1 --2- 172.21.15.8:0/3580053104 >> v2:172.21.15.46:3300/0 conn(0x7f3cd80c5f80 0x7f3cd80c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdea08640 1 -- 172.21.15.8:0/3580053104 >> v2:172.21.15.46:3300/0 conn(0x7f3cd80c5f80 msgr2=0x7f3cd80c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdea08640 1 --2- 172.21.15.8:0/3580053104 >> v2:172.21.15.46:3300/0 conn(0x7f3cd80c5f80 0x7f3cd80c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdea08640 1 -- 172.21.15.8:0/3580053104 >> v1:172.21.15.8:6789/0 conn(0x7f3cd80c5ba0 legacy=0x7f3cd8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdea08640 1 -- 172.21.15.8:0/3580053104 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd806f490 con 0x7f3cd80c9b40 2023-10-30T12:47:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdd205640 1 -- 172.21.15.8:0/3580053104 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417864359 0 0) 0x7f3cd815cda0 con 0x7f3cd80c5ba0 2023-10-30T12:47:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdea08640 1 --2- 172.21.15.8:0/3580053104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81597b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cd806e290 tx=0x7f3cd001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b0f800c0118fbc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdd205640 1 -- 172.21.15.8:0/3580053104 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00a57d0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdd205640 1 -- 172.21.15.8:0/3580053104 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00a5930 con 0x7f3cd80c9b40 2023-10-30T12:47:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3cdd205640 1 -- 172.21.15.8:0/3580053104 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00a5cc0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 -- 172.21.15.8:0/3580053104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 msgr2=0x7f3cd81597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3580053104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81597b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cd806e290 tx=0x7f3cd001c910 comp rx=0 tx=0).stop 2023-10-30T12:47:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 -- 172.21.15.8:0/3580053104 shutdown_connections 2023-10-30T12:47:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3580053104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81597b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3580053104 >> v2:172.21.15.46:3300/0 conn(0x7f3cd80c5f80 0x7f3cd80c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 -- 172.21.15.8:0/3580053104 >> 172.21.15.8:0/3580053104 conn(0x7f3cd806afa0 msgr2=0x7f3cd806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 -- 172.21.15.8:0/3580053104 shutdown_connections 2023-10-30T12:47:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.763+0000 7f3ce046b640 1 -- 172.21.15.8:0/3580053104 wait complete. 2023-10-30T12:47:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 Processor -- start 2023-10-30T12:47:35.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- start start 2023-10-30T12:47:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd80c5ba0 0x7f3cd815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3cd80c5f80 0x7f3cd8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81585a0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8156090 con 0x7f3cd80c5ba0 2023-10-30T12:47:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8155ab0 con 0x7f3cd80c5f80 2023-10-30T12:47:35.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47388/0 (socket says 172.21.15.8:47388) 2023-10-30T12:47:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 -- 172.21.15.8:0/3294715617 learned_addr learned my addr 172.21.15.8:0/3294715617 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cde207640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd80c5ba0 0x7f3cd815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdda06640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3cd80c5f80 0x7f3cd8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 -- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3cd80c5f80 msgr2=0x7f3cd8159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3cd80c5f80 0x7f3cd8159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 -- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd80c5ba0 msgr2=0x7f3cd815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd80c5ba0 0x7f3cd815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 -- 172.21.15.8:0/3294715617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0009a70 con 0x7f3cd80c9b40 2023-10-30T12:47:35.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cdea08640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009d90 tx=0x7f3cd00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00b7640 con 0x7f3cd80c9b40 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00b77a0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd81587b0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00b7b50 con 0x7f3cd80c9b40 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.767+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd8155d20 con 0x7f3cd80c9b40 2023-10-30T12:47:35.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd804e170 con 0x7f3cd80c9b40 2023-10-30T12:47:35.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3cd00b7cb0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3cceffd640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 0x7f3cbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:35.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f3cd011edc0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3cde207640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 0x7f3cbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:35.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.771+0000 7f3cde207640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 0x7f3cbc068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000f70 tx=0x7f3cd4007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:35.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.775+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3cd00ec1e0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.963+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd81561d0 con 0x7f3cd80c9b40 2023-10-30T12:47:35.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.963+0000 7f3cceffd640 1 -- 172.21.15.8:0/3294715617 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5728 (secure 0 0 0) 0x7f3cd00ef800 con 0x7f3cd80c9b40 2023-10-30T12:47:35.970 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:35.971 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:33.054806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:35.971 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 15 2023-10-30T12:47:35.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 msgr2=0x7f3cbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 0x7f3cbc068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000f70 tx=0x7f3cd4007c10 comp rx=0 tx=0).stop 2023-10-30T12:47:35.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 msgr2=0x7f3cd81596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:35.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009d90 tx=0x7f3cd00196b0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 shutdown_connections 2023-10-30T12:47:35.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cbc0664a0 0x7f3cbc068960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cd80c9b40 0x7f3cd81596d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3cd80c5f80 0x7f3cd8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 --2- 172.21.15.8:0/3294715617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd80c5ba0 0x7f3cd815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:35.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 >> 172.21.15.8:0/3294715617 conn(0x7f3cd806afa0 msgr2=0x7f3cd80733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:35.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 shutdown_connections 2023-10-30T12:47:35.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:35.967+0000 7f3ce046b640 1 -- 172.21.15.8:0/3294715617 wait complete. 2023-10-30T12:47:35.993 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 15, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 15, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4244": { "addr": "172.21.15.8:6838/2992596118", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 2992596118, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 2992596118, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 13, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 37 }, "gid_4276": { "addr": "172.21.15.46:6833/3888673612", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 3888673612, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 3888673612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4276, "incarnation": 9, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:47:33.054806+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 4276, "mds_2": 4244 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:47:37.995 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:38.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 Processor -- start 2023-10-30T12:47:38.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 -- start start 2023-10-30T12:47:38.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f225c06b760 0x7f225c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c071e00 con 0x7f225c06c420 2023-10-30T12:47:38.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c070020 con 0x7f225c06bb40 2023-10-30T12:47:38.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f226346b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225c15eb90 con 0x7f225c06b760 2023-10-30T12:47:38.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261a08640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f225c06c420 0x7f225c0c57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40354/0 (socket says 172.21.15.8:40354) 2023-10-30T12:47:38.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261207640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f225c06b760 0x7f225c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261a08640 1 -- 172.21.15.8:0/1212731664 learned_addr learned my addr 172.21.15.8:0/1212731664 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:38.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261207640 1 -- 172.21.15.8:0/1212731664 >> v1:172.21.15.46:6789/0 conn(0x7f225c06bb40 legacy=0x7f225c157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261207640 1 -- 172.21.15.8:0/1212731664 >> v1:172.21.15.8:6789/0 conn(0x7f225c06c420 legacy=0x7f225c0c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2261207640 1 -- 172.21.15.8:0/1212731664 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c070f10 con 0x7f225c06b760 2023-10-30T12:47:38.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.147+0000 7f2253fff640 1 -- 172.21.15.8:0/1212731664 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521965123 0 0) 0x7f225c071e00 con 0x7f225c06c420 2023-10-30T12:47:38.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f2261207640 1 --2- 172.21.15.8:0/1212731664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06b760 0x7f225c15e650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22580097c0 tx=0x7f225801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29944066e20764d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f2253fff640 1 -- 172.21.15.8:0/1212731664 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580ab650 con 0x7f225c06b760 2023-10-30T12:47:38.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f2253fff640 1 -- 172.21.15.8:0/1212731664 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22580ab7b0 con 0x7f225c06b760 2023-10-30T12:47:38.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f2253fff640 1 -- 172.21.15.8:0/1212731664 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580abb40 con 0x7f225c06b760 2023-10-30T12:47:38.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 -- 172.21.15.8:0/1212731664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06b760 msgr2=0x7f225c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 --2- 172.21.15.8:0/1212731664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06b760 0x7f225c15e650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22580097c0 tx=0x7f225801c450 comp rx=0 tx=0).stop 2023-10-30T12:47:38.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 -- 172.21.15.8:0/1212731664 shutdown_connections 2023-10-30T12:47:38.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 --2- 172.21.15.8:0/1212731664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06b760 0x7f225c15e650 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 -- 172.21.15.8:0/1212731664 >> 172.21.15.8:0/1212731664 conn(0x7f225c0cac70 msgr2=0x7f225c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:38.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 -- 172.21.15.8:0/1212731664 shutdown_connections 2023-10-30T12:47:38.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.151+0000 7f226346b640 1 -- 172.21.15.8:0/1212731664 wait complete. 2023-10-30T12:47:38.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 Processor -- start 2023-10-30T12:47:38.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 -- start start 2023-10-30T12:47:38.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f225c06b760 0x7f225c0c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225c06bb40 0x7f225c0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c14a640 con 0x7f225c06c420 2023-10-30T12:47:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c15cdc0 con 0x7f225c06bb40 2023-10-30T12:47:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c14a0c0 con 0x7f225c06b760 2023-10-30T12:47:38.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261207640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f225c06b760 0x7f225c0c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47414/0 (socket says 172.21.15.8:47414) 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 -- 172.21.15.8:0/2761682355 learned_addr learned my addr 172.21.15.8:0/2761682355 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2260a06640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225c06bb40 0x7f225c0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 -- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f225c06b760 msgr2=0x7f225c0c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f225c06b760 0x7f225c0c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 -- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225c06bb40 msgr2=0x7f225c0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225c06bb40 0x7f225c0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 -- 172.21.15.8:0/2761682355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258009420 con 0x7f225c06c420 2023-10-30T12:47:38.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2261a08640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f225400c700 tx=0x7f225400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400ce10 con 0x7f225c06c420 2023-10-30T12:47:38.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225400a640 con 0x7f225c06c420 2023-10-30T12:47:38.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400aa30 con 0x7f225c06c420 2023-10-30T12:47:38.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.155+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c14a360 con 0x7f225c06c420 2023-10-30T12:47:38.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c0c4370 con 0x7f225c06c420 2023-10-30T12:47:38.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225c15cf70 con 0x7f225c06c420 2023-10-30T12:47:38.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f225400ab90 con 0x7f225c06c420 2023-10-30T12:47:38.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f2251ffb640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 0x7f2240068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:38.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f2254083410 con 0x7f225c06c420 2023-10-30T12:47:38.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f2261207640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 0x7f2240068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:38.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.159+0000 7f2261207640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 0x7f2240068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22580af000 tx=0x7f22580a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:38.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.163+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f225c15cf70 con 0x7f225c06c420 2023-10-30T12:47:38.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.355+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f225c15cf30 con 0x7f225c06c420 2023-10-30T12:47:38.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.355+0000 7f2251ffb640 1 -- 172.21.15.8:0/2761682355 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f2254018020 con 0x7f225c06c420 2023-10-30T12:47:38.362 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:38.363 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:38.073469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:38.363 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 16 2023-10-30T12:47:38.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 msgr2=0x7f2240068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 0x7f2240068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22580af000 tx=0x7f22580a7000 comp rx=0 tx=0).stop 2023-10-30T12:47:38.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 msgr2=0x7f225c1f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:38.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f225400c700 tx=0x7f225400cbc0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 shutdown_connections 2023-10-30T12:47:38.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22400664a0 0x7f2240068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225c06bb40 0x7f225c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f225c06c420 0x7f225c1f9c00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 --2- 172.21.15.8:0/2761682355 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f225c06b760 0x7f225c0c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 >> 172.21.15.8:0/2761682355 conn(0x7f225c0cac70 msgr2=0x7f225c153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:38.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 shutdown_connections 2023-10-30T12:47:38.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:38.359+0000 7f226346b640 1 -- 172.21.15.8:0/2761682355 wait complete. 2023-10-30T12:47:38.384 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:40.385 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 Processor -- start 2023-10-30T12:47:40.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 -- start start 2023-10-30T12:47:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1094153c80 0x7f109415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10940c4a10 0x7f10940c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f109415a3b0 0x7f10940ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109415dec0 con 0x7f109415a3b0 2023-10-30T12:47:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10941599a0 con 0x7f10940c4a10 2023-10-30T12:47:40.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109c0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1094159f20 con 0x7f1094153c80 2023-10-30T12:47:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f109415a3b0 0x7f10940ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f109415a3b0 0x7f10940ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55928/0 (socket says 172.21.15.8:55928) 2023-10-30T12:47:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 -- 172.21.15.8:0/3233702396 learned_addr learned my addr 172.21.15.8:0/3233702396 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f1099e65640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1094153c80 0x7f109415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f1099664640 1 --2- 172.21.15.8:0/3233702396 >> v2:172.21.15.46:3300/0 conn(0x7f10940c4a10 0x7f10940c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 -- 172.21.15.8:0/3233702396 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1094153c80 msgr2=0x7f109415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1094153c80 0x7f109415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 -- 172.21.15.8:0/3233702396 >> v2:172.21.15.46:3300/0 conn(0x7f10940c4a10 msgr2=0x7f10940c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:40.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 --2- 172.21.15.8:0/3233702396 >> v2:172.21.15.46:3300/0 conn(0x7f10940c4a10 0x7f10940c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.535+0000 7f109a666640 1 -- 172.21.15.8:0/3233702396 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940c47e0 con 0x7f109415a3b0 2023-10-30T12:47:40.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109a666640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f109415a3b0 0x7f10940ca1e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f109408cc20 tx=0x7f108c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26fdcf5326af839 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f1098e63640 1 -- 172.21.15.8:0/3233702396 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0af400 con 0x7f109415a3b0 2023-10-30T12:47:40.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f1098e63640 1 -- 172.21.15.8:0/3233702396 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c0ab050 con 0x7f109415a3b0 2023-10-30T12:47:40.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f1098e63640 1 -- 172.21.15.8:0/3233702396 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0af780 con 0x7f109415a3b0 2023-10-30T12:47:40.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 -- 172.21.15.8:0/3233702396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f109415a3b0 msgr2=0x7f10940ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f109415a3b0 0x7f10940ca1e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f109408cc20 tx=0x7f108c01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 -- 172.21.15.8:0/3233702396 shutdown_connections 2023-10-30T12:47:40.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f109415a3b0 0x7f10940ca1e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3233702396 >> v2:172.21.15.46:3300/0 conn(0x7f10940c4a10 0x7f10940c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3233702396 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1094153c80 0x7f109415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 -- 172.21.15.8:0/3233702396 >> 172.21.15.8:0/3233702396 conn(0x7f10940715c0 msgr2=0x7f10940719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:40.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 -- 172.21.15.8:0/3233702396 shutdown_connections 2023-10-30T12:47:40.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.539+0000 7f109c0c9640 1 -- 172.21.15.8:0/3233702396 wait complete. 2023-10-30T12:47:40.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 Processor -- start 2023-10-30T12:47:40.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- start start 2023-10-30T12:47:40.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1094153c80 0x7f10941ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f109415a3b0 0x7f109414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109414ad60 con 0x7f10940c4a10 2023-10-30T12:47:40.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10941ef000 con 0x7f1094153c80 2023-10-30T12:47:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940ce3b0 con 0x7f109415a3b0 2023-10-30T12:47:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55944/0 (socket says 172.21.15.8:55944) 2023-10-30T12:47:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 -- 172.21.15.8:0/3844016432 learned_addr learned my addr 172.21.15.8:0/3844016432 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:40.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109a666640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f109415a3b0 0x7f109414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099664640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1094153c80 0x7f10941ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 -- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f109415a3b0 msgr2=0x7f109414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f109415a3b0 0x7f109414f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 -- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1094153c80 msgr2=0x7f10941ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1094153c80 0x7f10941ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 -- 172.21.15.8:0/3844016432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c009b30 con 0x7f10940c4a10 2023-10-30T12:47:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f1099e65640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f109000d800 tx=0x7f109000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000b6c0 con 0x7f10940c4a10 2023-10-30T12:47:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000b820 con 0x7f10940c4a10 2023-10-30T12:47:40.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000bb90 con 0x7f10940c4a10 2023-10-30T12:47:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10941f2f20 con 0x7f10940c4a10 2023-10-30T12:47:40.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.543+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10941f2960 con 0x7f10940c4a10 2023-10-30T12:47:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109404e170 con 0x7f10940c4a10 2023-10-30T12:47:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f109000bcf0 con 0x7f10940c4a10 2023-10-30T12:47:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f108a7fc640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 0x7f1078068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f1090084670 con 0x7f10940c4a10 2023-10-30T12:47:40.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f1099664640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 0x7f1078068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:40.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.547+0000 7f1099664640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 0x7f1078068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1094158b60 tx=0x7f1084008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:40.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.551+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10900518d0 con 0x7f10940c4a10 2023-10-30T12:47:40.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.739+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10940cad40 con 0x7f10940c4a10 2023-10-30T12:47:40.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f108a7fc640 1 -- 172.21.15.8:0/3844016432 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f1090054ef0 con 0x7f10940c4a10 2023-10-30T12:47:40.747 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:40.748 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:38.073469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:40.748 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 16 2023-10-30T12:47:40.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 msgr2=0x7f1078068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 0x7f1078068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1094158b60 tx=0x7f1084008040 comp rx=0 tx=0).stop 2023-10-30T12:47:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 msgr2=0x7f10941f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f109000d800 tx=0x7f109000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 shutdown_connections 2023-10-30T12:47:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f10780664a0 0x7f1078068960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1094153c80 0x7f10941ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f109415a3b0 0x7f109414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 --2- 172.21.15.8:0/3844016432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10940c4a10 0x7f10941f2030 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:40.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 >> 172.21.15.8:0/3844016432 conn(0x7f10940715c0 msgr2=0x7f10940c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:40.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 shutdown_connections 2023-10-30T12:47:40.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:40.743+0000 7f109c0c9640 1 -- 172.21.15.8:0/3844016432 wait complete. 2023-10-30T12:47:40.769 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:47:42.771 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:42.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 Processor -- start 2023-10-30T12:47:42.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 -- start start 2023-10-30T12:47:42.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf180c8060 0x7fdf1806f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf180ca9b0 0x7fdf1815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180ca180 con 0x7fdf180cb8b0 2023-10-30T12:47:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf18150d00 con 0x7fdf180c8060 2023-10-30T12:47:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.919+0000 7fdf1f73e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf180ca740 con 0x7fdf180ca9b0 2023-10-30T12:47:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf180c8060 0x7fdf1806f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1ccd9640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fdf180cb8b0 0x7fdf180c8730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53414/0 (socket says 172.21.15.8:53414) 2023-10-30T12:47:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1ccd9640 1 -- 172.21.15.8:0/3065661577 learned_addr learned my addr 172.21.15.8:0/3065661577 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1dcdb640 1 --2- 172.21.15.8:0/3065661577 >> v2:172.21.15.46:3300/0 conn(0x7fdf180ca9b0 0x7fdf1815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 -- 172.21.15.8:0/3065661577 >> v2:172.21.15.46:3300/0 conn(0x7fdf180ca9b0 msgr2=0x7fdf1815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 --2- 172.21.15.8:0/3065661577 >> v2:172.21.15.46:3300/0 conn(0x7fdf180ca9b0 0x7fdf1815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 -- 172.21.15.8:0/3065661577 >> v1:172.21.15.8:6789/0 conn(0x7fdf180cb8b0 legacy=0x7fdf180c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf0ffff640 1 -- 172.21.15.8:0/3065661577 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110277529 0 0) 0x7fdf180ca180 con 0x7fdf180cb8b0 2023-10-30T12:47:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 -- 172.21.15.8:0/3065661577 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180ca460 con 0x7fdf180c8060 2023-10-30T12:47:42.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf1d4da640 1 --2- 172.21.15.8:0/3065661577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180c8060 0x7fdf1806f290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf1806d720 tx=0x7fdf1401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e80c77e3750cd9a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:42.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf0ffff640 1 -- 172.21.15.8:0/3065661577 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140a8950 con 0x7fdf180c8060 2023-10-30T12:47:42.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf0ffff640 1 -- 172.21.15.8:0/3065661577 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf140a8ab0 con 0x7fdf180c8060 2023-10-30T12:47:42.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.923+0000 7fdf0ffff640 1 -- 172.21.15.8:0/3065661577 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140a8e40 con 0x7fdf180c8060 2023-10-30T12:47:42.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- 172.21.15.8:0/3065661577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180c8060 msgr2=0x7fdf1806f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:42.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/3065661577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180c8060 0x7fdf1806f290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf1806d720 tx=0x7fdf1401ec50 comp rx=0 tx=0).stop 2023-10-30T12:47:42.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- 172.21.15.8:0/3065661577 shutdown_connections 2023-10-30T12:47:42.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/3065661577 >> v2:172.21.15.46:3300/0 conn(0x7fdf180ca9b0 0x7fdf1815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:42.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/3065661577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180c8060 0x7fdf1806f290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:42.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- 172.21.15.8:0/3065661577 >> 172.21.15.8:0/3065661577 conn(0x7fdf1806afa0 msgr2=0x7fdf1806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:42.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- 172.21.15.8:0/3065661577 shutdown_connections 2023-10-30T12:47:42.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- 172.21.15.8:0/3065661577 wait complete. 2023-10-30T12:47:42.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 Processor -- start 2023-10-30T12:47:42.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- start start 2023-10-30T12:47:42.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf180c8060 0x7fdf18152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 0x7fdf18155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18153870 con 0x7fdf180ca9b0 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181528a0 con 0x7fdf180c8060 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1f73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18157140 con 0x7fdf180cb8b0 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1ccd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1ccd9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55968/0 (socket says 172.21.15.8:55968) 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1ccd9640 1 -- 172.21.15.8:0/2879306153 learned_addr learned my addr 172.21.15.8:0/2879306153 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:42.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1d4da640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf180c8060 0x7fdf18152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.927+0000 7fdf1dcdb640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 0x7fdf18155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 -- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 msgr2=0x7fdf18155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:42.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 0x7fdf18155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:42.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 -- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf180c8060 msgr2=0x7fdf18152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:42.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf180c8060 0x7fdf18152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:42.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 -- 172.21.15.8:0/2879306153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14009420 con 0x7fdf180ca9b0 2023-10-30T12:47:42.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1dcdb640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 0x7fdf18155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:42.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1ccd9640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c4b0 tx=0x7fdf0800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:42.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080074e0 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf08007660 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004a30 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf18156e60 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf18152250 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1814bb50 con 0x7fdf180ca9b0 2023-10-30T12:47:42.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf08004b90 con 0x7fdf180ca9b0 2023-10-30T12:47:42.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 0x7fdefc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:42.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdf08082fe0 con 0x7fdf180ca9b0 2023-10-30T12:47:42.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.931+0000 7fdf1d4da640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 0x7fdefc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:42.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.935+0000 7fdf1d4da640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 0x7fdefc068990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b2000 tx=0x7fdf140aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:42.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:42.935+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf080503e0 con 0x7fdf180ca9b0 2023-10-30T12:47:43.128 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 169 2023-10-30T12:47:43.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.135+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf1804e170 con 0x7fdf180ca9b0 2023-10-30T12:47:43.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.135+0000 7fdf0dffb640 1 -- 172.21.15.8:0/2879306153 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5725 (secure 0 0 0) 0x7fdf08053a00 con 0x7fdf180ca9b0 2023-10-30T12:47:43.142 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:43.142 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:43.116709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:43.143 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2023-10-30T12:47:43.143 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 170 2023-10-30T12:47:43.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 msgr2=0x7fdefc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 0x7fdefc068990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b2000 tx=0x7fdf140aa070 comp rx=0 tx=0).stop 2023-10-30T12:47:43.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 msgr2=0x7fdf180c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:43.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c4b0 tx=0x7fdf0800c970 comp rx=0 tx=0).stop 2023-10-30T12:47:43.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 shutdown_connections 2023-10-30T12:47:43.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdefc0664d0 0x7fdefc068990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf180cb8b0 0x7fdf18155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf180ca9b0 0x7fdf180c4b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 --2- 172.21.15.8:0/2879306153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf180c8060 0x7fdf18152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:43.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 >> 172.21.15.8:0/2879306153 conn(0x7fdf1806afa0 msgr2=0x7fdf18150450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:43.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 shutdown_connections 2023-10-30T12:47:43.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:43.139+0000 7fdf1f73e640 1 -- 172.21.15.8:0/2879306153 wait complete. 2023-10-30T12:47:43.154 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 171 2023-10-30T12:47:43.165 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 172 2023-10-30T12:47:43.169 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=4247 2023-10-30T12:47:43.170 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2023-10-30T12:47:43.177 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 173 2023-10-30T12:47:43.188 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 174 2023-10-30T12:47:43.199 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 175 2023-10-30T12:47:43.211 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 176 2023-10-30T12:47:43.223 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 177 2023-10-30T12:47:43.368 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 178 2023-10-30T12:47:43.380 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 179 2023-10-30T12:47:43.392 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 180 2023-10-30T12:47:43.404 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 181 2023-10-30T12:47:43.416 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 182 2023-10-30T12:47:43.427 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 183 2023-10-30T12:47:43.439 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 184 2023-10-30T12:47:43.451 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 185 2023-10-30T12:47:43.747 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 186 2023-10-30T12:47:43.759 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 187 2023-10-30T12:47:43.771 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 188 2023-10-30T12:47:43.783 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 189 2023-10-30T12:47:43.795 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 190 2023-10-30T12:47:43.807 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 191 2023-10-30T12:47:43.818 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 192 2023-10-30T12:47:43.830 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 193 2023-10-30T12:47:43.842 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 194 2023-10-30T12:47:43.853 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 195 2023-10-30T12:47:43.865 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 196 2023-10-30T12:47:43.877 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 197 2023-10-30T12:47:43.888 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 198 2023-10-30T12:47:43.900 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 199 2023-10-30T12:47:44.169 INFO:tasks.workunit.client.0.smithi008.stdout:Take first snapshot .snap/test1 2023-10-30T12:47:44.192 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:44.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:47:48.132 INFO:tasks.workunit.client.0.smithi008.stdout:Create dir 200 to 299 ... 2023-10-30T12:47:48.134 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 200 2023-10-30T12:47:48.142 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 201 2023-10-30T12:47:48.156 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 202 2023-10-30T12:47:48.168 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 203 2023-10-30T12:47:48.170 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-30T12:47:48.171 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:47:48.171 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:47:48.177 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:47:48.177 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:48.182 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 204 2023-10-30T12:47:48.195 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 205 2023-10-30T12:47:48.208 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 206 2023-10-30T12:47:48.222 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 207 2023-10-30T12:47:48.235 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 208 2023-10-30T12:47:48.248 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 209 2023-10-30T12:47:48.258 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 210 2023-10-30T12:47:48.273 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 211 2023-10-30T12:47:48.275 INFO:tasks.ceph.mds.c.smithi008.stdout:starting mds.c at 2023-10-30T12:47:48.303 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 212 2023-10-30T12:47:48.314 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 213 2023-10-30T12:47:48.326 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 214 2023-10-30T12:47:48.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.347+0000 7f68e309c640 1 Processor -- start 2023-10-30T12:47:48.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.347+0000 7f68e309c640 1 -- start start 2023-10-30T12:47:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e309c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f68dc070520 0x7f68dc070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e309c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68dc0c2310 con 0x7f68dc150830 2023-10-30T12:47:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e309c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68dc156710 con 0x7f68dc0cb7d0 2023-10-30T12:47:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e309c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68dc06bc10 con 0x7f68dc070520 2023-10-30T12:47:48.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e0e38640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f68dc150830 0x7f68dc06b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:53426/0 (socket says 172.21.15.8:53426) 2023-10-30T12:47:48.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e0e38640 1 -- 172.21.15.8:0/318532977 learned_addr learned my addr 172.21.15.8:0/318532977 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:47:48.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e1639640 1 --2- 172.21.15.8:0/318532977 >> v2:172.21.15.8:3300/0 conn(0x7f68dc070520 0x7f68dc070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68db7fe640 1 -- 172.21.15.8:0/318532977 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671620645 0 0) 0x7f68dc0c2310 con 0x7f68dc150830 2023-10-30T12:47:48.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68db7fe640 1 -- 172.21.15.8:0/318532977 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68cc005740 con 0x7f68dc150830 2023-10-30T12:47:48.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e1639640 1 -- 172.21.15.8:0/318532977 >> v1:172.21.15.46:6789/0 conn(0x7f68dc0cb7d0 legacy=0x7f68dc0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e1639640 1 -- 172.21.15.8:0/318532977 >> v1:172.21.15.8:6789/0 conn(0x7f68dc150830 legacy=0x7f68dc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.351+0000 7f68e1639640 1 -- 172.21.15.8:0/318532977 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc0c4360 con 0x7f68dc070520 2023-10-30T12:47:48.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e1639640 1 --2- 172.21.15.8:0/318532977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc070520 0x7f68dc070990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f37cd2cf421b475 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68db7fe640 1 -- 172.21.15.8:0/318532977 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40af600 con 0x7f68dc070520 2023-10-30T12:47:48.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68db7fe640 1 -- 172.21.15.8:0/318532977 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d40ab050 con 0x7f68dc070520 2023-10-30T12:47:48.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68db7fe640 1 -- 172.21.15.8:0/318532977 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40af980 con 0x7f68dc070520 2023-10-30T12:47:48.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- 172.21.15.8:0/318532977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc070520 msgr2=0x7f68dc070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 --2- 172.21.15.8:0/318532977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc070520 0x7f68dc070990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d401f130 comp rx=0 tx=0).stop 2023-10-30T12:47:48.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- 172.21.15.8:0/318532977 shutdown_connections 2023-10-30T12:47:48.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 --2- 172.21.15.8:0/318532977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc070520 0x7f68dc070990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- 172.21.15.8:0/318532977 >> 172.21.15.8:0/318532977 conn(0x7f68dc0cac70 msgr2=0x7f68dc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:48.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- 172.21.15.8:0/318532977 shutdown_connections 2023-10-30T12:47:48.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- 172.21.15.8:0/318532977 wait complete. 2023-10-30T12:47:48.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 Processor -- start 2023-10-30T12:47:48.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.355+0000 7f68e309c640 1 -- start start 2023-10-30T12:47:48.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68dc070520 0x7f68dc149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 0x7f68dc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc148240 con 0x7f68dc150830 2023-10-30T12:47:48.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc156210 con 0x7f68dc0cb7d0 2023-10-30T12:47:48.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc1548d0 con 0x7f68dc070520 2023-10-30T12:47:48.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55988/0 (socket says 172.21.15.8:55988) 2023-10-30T12:47:48.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 -- 172.21.15.8:0/1550961481 learned_addr learned my addr 172.21.15.8:0/1550961481 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e0e38640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68dc070520 0x7f68dc149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68dbfff640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 0x7f68dc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 -- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68dc070520 msgr2=0x7f68dc149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68dc070520 0x7f68dc149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 -- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 msgr2=0x7f68dc154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 0x7f68dc154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 -- 172.21.15.8:0/1550961481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009a70 con 0x7f68dc150830 2023-10-30T12:47:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68dbfff640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 0x7f68dc154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:47:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e1639640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d401bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4009450 con 0x7f68dc150830 2023-10-30T12:47:48.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d40b9410 con 0x7f68dc150830 2023-10-30T12:47:48.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc1542c0 con 0x7f68dc150830 2023-10-30T12:47:48.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40b97c0 con 0x7f68dc150830 2023-10-30T12:47:48.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc153ba0 con 0x7f68dc150830 2023-10-30T12:47:48.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68dc066510 con 0x7f68dc150830 2023-10-30T12:47:48.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.359+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68d40b6050 con 0x7f68dc150830 2023-10-30T12:47:48.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.363+0000 7f68d97fa640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 0x7f68c0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:48.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.363+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f68d41224c0 con 0x7f68dc150830 2023-10-30T12:47:48.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.363+0000 7f68e0e38640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 0x7f68c0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:48.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.363+0000 7f68e0e38640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 0x7f68c0068990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005890 tx=0x7f68d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:48.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.367+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68d40ef7b0 con 0x7f68dc150830 2023-10-30T12:47:48.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68dc154a60 con 0x7f68dc150830 2023-10-30T12:47:48.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68d97fa640 1 -- 172.21.15.8:0/1550961481 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f68d40f2dd0 con 0x7f68dc150830 2023-10-30T12:47:48.564 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:48.564 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:43.116709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:48.564 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2023-10-30T12:47:48.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 msgr2=0x7f68c0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 0x7f68c0068990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005890 tx=0x7f68d0009040 comp rx=0 tx=0).stop 2023-10-30T12:47:48.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 msgr2=0x7f68dc06bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:48.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d401bed0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 shutdown_connections 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68c00664d0 0x7f68c0068990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68dc0cb7d0 0x7f68dc154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68dc150830 0x7f68dc06bfb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 --2- 172.21.15.8:0/1550961481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68dc070520 0x7f68dc149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:48.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 >> 172.21.15.8:0/1550961481 conn(0x7f68dc0cac70 msgr2=0x7f68dc0c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:48.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 shutdown_connections 2023-10-30T12:47:48.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:48.559+0000 7f68e309c640 1 -- 172.21.15.8:0/1550961481 wait complete. 2023-10-30T12:47:48.586 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-30T12:47:48.623 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 215 2023-10-30T12:47:48.635 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 216 2023-10-30T12:47:48.645 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 217 2023-10-30T12:47:48.656 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 218 2023-10-30T12:47:48.667 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 219 2023-10-30T12:47:48.678 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 220 2023-10-30T12:47:48.690 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 221 2023-10-30T12:47:48.700 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 222 2023-10-30T12:47:48.710 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 223 2023-10-30T12:47:48.720 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 224 2023-10-30T12:47:48.731 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 225 2023-10-30T12:47:48.741 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 226 2023-10-30T12:47:48.752 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 227 2023-10-30T12:47:48.762 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 228 2023-10-30T12:47:49.420 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 229 2023-10-30T12:47:49.431 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 230 2023-10-30T12:47:49.441 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 231 2023-10-30T12:47:49.452 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 232 2023-10-30T12:47:49.714 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 233 2023-10-30T12:47:49.725 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 234 2023-10-30T12:47:49.736 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 235 2023-10-30T12:47:49.748 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 236 2023-10-30T12:47:49.758 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 237 2023-10-30T12:47:49.769 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 238 2023-10-30T12:47:49.779 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 239 2023-10-30T12:47:49.791 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 240 2023-10-30T12:47:49.802 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 241 2023-10-30T12:47:49.813 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 242 2023-10-30T12:47:49.824 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 243 2023-10-30T12:47:49.835 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 244 2023-10-30T12:47:50.172 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 245 2023-10-30T12:47:50.182 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 246 2023-10-30T12:47:50.193 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 247 2023-10-30T12:47:50.203 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 248 2023-10-30T12:47:50.213 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 249 2023-10-30T12:47:50.473 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 250 2023-10-30T12:47:50.502 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 251 2023-10-30T12:47:50.587 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:47:50.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.723+0000 7fa59149e640 1 Processor -- start 2023-10-30T12:47:50.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.723+0000 7fa59149e640 1 -- start start 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa59149e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c1537f0 0x7fa58c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa59149e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa58c06b2f0 0x7fa58c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa59149e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58c0c2aa0 con 0x7fa58c15a2e0 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa59149e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58c0c3ab0 con 0x7fa58c06b2f0 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa59149e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0706e0 con 0x7fa58c1537f0 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa58c06b2f0 0x7fa58c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa58c06b2f0 0x7fa58c06b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34014/0 (socket says 172.21.15.8:34014) 2023-10-30T12:47:50.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 -- 172.21.15.8:0/1118058430 learned_addr learned my addr 172.21.15.8:0/1118058430 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58affd640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c1537f0 0x7fa58c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 -- 172.21.15.8:0/1118058430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c1537f0 msgr2=0x7fa58c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c1537f0 0x7fa58c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 -- 172.21.15.8:0/1118058430 >> v1:172.21.15.46:6789/0 conn(0x7fa58c15a2e0 legacy=0x7fa58c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.727+0000 7fa58b7fe640 1 -- 172.21.15.8:0/1118058430 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c06bca0 con 0x7fa58c06b2f0 2023-10-30T12:47:50.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa58b7fe640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c06b2f0 0x7fa58c06b760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa58401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715151e35e6ad2be server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa58a7fc640 1 -- 172.21.15.8:0/1118058430 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840af400 con 0x7fa58c06b2f0 2023-10-30T12:47:50.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa58a7fc640 1 -- 172.21.15.8:0/1118058430 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840ab050 con 0x7fa58c06b2f0 2023-10-30T12:47:50.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa58a7fc640 1 -- 172.21.15.8:0/1118058430 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840af780 con 0x7fa58c06b2f0 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- 172.21.15.8:0/1118058430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c06b2f0 msgr2=0x7fa58c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c06b2f0 0x7fa58c06b760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa58401f1f0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- 172.21.15.8:0/1118058430 shutdown_connections 2023-10-30T12:47:50.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c06b2f0 0x7fa58c06b760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 --2- 172.21.15.8:0/1118058430 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c1537f0 0x7fa58c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- 172.21.15.8:0/1118058430 >> 172.21.15.8:0/1118058430 conn(0x7fa58c071fb0 msgr2=0x7fa58c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:50.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- 172.21.15.8:0/1118058430 shutdown_connections 2023-10-30T12:47:50.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- 172.21.15.8:0/1118058430 wait complete. 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 Processor -- start 2023-10-30T12:47:50.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.731+0000 7fa59149e640 1 -- start start 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58c06b2f0 0x7fa58c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c15a2e0 0x7fa58c15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1f10e0 con 0x7fa58c1537f0 2023-10-30T12:47:50.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1f0210 con 0x7fa58c06b2f0 2023-10-30T12:47:50.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1f0460 con 0x7fa58c15a2e0 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34024/0 (socket says 172.21.15.8:34024) 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 -- 172.21.15.8:0/3697563889 learned_addr learned my addr 172.21.15.8:0/3697563889 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa58affd640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58c06b2f0 0x7fa58c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa58b7fe640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c15a2e0 0x7fa58c15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 -- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c15a2e0 msgr2=0x7fa58c15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c15a2e0 0x7fa58c15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 -- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58c06b2f0 msgr2=0x7fa58c1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58c06b2f0 0x7fa58c1596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 -- 172.21.15.8:0/3697563889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009b30 con 0x7fa58c1537f0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa583fff640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c500 tx=0x7fa57400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574005220 con 0x7fa58c1537f0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa574005380 con 0x7fa58c1537f0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574007270 con 0x7fa58c1537f0 2023-10-30T12:47:50.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c1f0740 con 0x7fa58c1537f0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c1f0d00 con 0x7fa58c1537f0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c1f0e90 con 0x7fa58c1537f0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.735+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa5740073d0 con 0x7fa58c1537f0 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.739+0000 7fa5837fe640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 0x7fa56c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:47:50.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.739+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fa574083780 con 0x7fa58c1537f0 2023-10-30T12:47:50.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.739+0000 7fa58affd640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 0x7fa56c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:47:50.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.739+0000 7fa58affd640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 0x7fa56c068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009b60 tx=0x7fa57c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:47:50.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.743+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa574050b30 con 0x7fa58c1537f0 2023-10-30T12:47:50.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa58c04e170 con 0x7fa58c1537f0 2023-10-30T12:47:50.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa5837fe640 1 -- 172.21.15.8:0/3697563889 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6513 (secure 0 0 0) 0x7fa574054150 con 0x7fa58c1537f0 2023-10-30T12:47:50.936 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:47:50.937 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:43.116709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:47:50.937 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 18 2023-10-30T12:47:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 msgr2=0x7fa56c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 0x7fa56c068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009b60 tx=0x7fa57c009190 comp rx=0 tx=0).stop 2023-10-30T12:47:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 msgr2=0x7fa58c15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:47:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c500 tx=0x7fa57400c9c0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 shutdown_connections 2023-10-30T12:47:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa56c0664a0 0x7fa56c068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa58c15a2e0 0x7fa58c15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa58c1537f0 0x7fa58c15d3d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 --2- 172.21.15.8:0/3697563889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58c06b2f0 0x7fa58c1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:47:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 >> 172.21.15.8:0/3697563889 conn(0x7fa58c071fb0 msgr2=0x7fa58c06c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:47:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 shutdown_connections 2023-10-30T12:47:50.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:47:50.931+0000 7fa59149e640 1 -- 172.21.15.8:0/3697563889 wait complete. 2023-10-30T12:47:50.957 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-30T12:47:50.957 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 81 secs before thrashing 2023-10-30T12:47:53.988 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 252 2023-10-30T12:47:53.999 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 253 2023-10-30T12:47:54.010 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 254 2023-10-30T12:47:54.020 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 255 2023-10-30T12:47:54.195 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 256 2023-10-30T12:47:54.228 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 257 2023-10-30T12:47:54.258 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 258 2023-10-30T12:47:54.290 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 259 2023-10-30T12:47:54.321 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 260 2023-10-30T12:47:55.020 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 261 2023-10-30T12:47:57.247 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 262 2023-10-30T12:47:57.260 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 263 2023-10-30T12:47:57.272 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 264 2023-10-30T12:47:57.284 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 265 2023-10-30T12:47:57.295 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 266 2023-10-30T12:47:57.307 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 267 2023-10-30T12:47:57.318 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 268 2023-10-30T12:47:57.336 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 269 2023-10-30T12:47:57.352 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 270 2023-10-30T12:47:57.365 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 271 2023-10-30T12:47:57.377 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 272 2023-10-30T12:47:57.388 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 273 2023-10-30T12:47:57.399 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 274 2023-10-30T12:47:57.410 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 275 2023-10-30T12:47:58.352 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 276 2023-10-30T12:47:58.366 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 277 2023-10-30T12:47:58.378 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 278 2023-10-30T12:47:58.390 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 279 2023-10-30T12:47:58.402 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 280 2023-10-30T12:47:58.664 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 281 2023-10-30T12:47:58.675 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 282 2023-10-30T12:47:58.686 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 283 2023-10-30T12:47:58.697 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 284 2023-10-30T12:48:00.063 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 285 2023-10-30T12:48:00.077 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 286 2023-10-30T12:48:00.110 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 287 2023-10-30T12:48:00.216 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 288 2023-10-30T12:48:00.236 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 289 2023-10-30T12:48:00.255 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 290 2023-10-30T12:48:00.275 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 291 2023-10-30T12:48:00.295 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 292 2023-10-30T12:48:00.564 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 293 2023-10-30T12:48:00.588 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 294 2023-10-30T12:48:00.617 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 295 2023-10-30T12:48:00.646 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 296 2023-10-30T12:48:00.676 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 297 2023-10-30T12:48:00.700 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 298 2023-10-30T12:48:00.724 INFO:tasks.workunit.client.0.smithi008.stdout: create dir 299 2023-10-30T12:48:00.790 INFO:tasks.workunit.client.0.smithi008.stdout:Create a snapshot in every first level dir ... 2023-10-30T12:48:01.054 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/.snap/snap-subdir-test 2023-10-30T12:48:01.082 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/30 file after the snapshot 2023-10-30T12:48:01.083 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/31 file after the snapshot 2023-10-30T12:48:01.084 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/32 file after the snapshot 2023-10-30T12:48:01.085 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/33 file after the snapshot 2023-10-30T12:48:01.085 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/34 file after the snapshot 2023-10-30T12:48:01.086 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/35 file after the snapshot 2023-10-30T12:48:01.087 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/36 file after the snapshot 2023-10-30T12:48:01.088 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/37 file after the snapshot 2023-10-30T12:48:01.088 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/38 file after the snapshot 2023-10-30T12:48:01.089 INFO:tasks.workunit.client.0.smithi008.stdout: create 100/39 file after the snapshot 2023-10-30T12:48:01.090 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/.snap/snap-subdir-test 2023-10-30T12:48:01.108 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/30 file after the snapshot 2023-10-30T12:48:01.113 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/31 file after the snapshot 2023-10-30T12:48:01.114 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/32 file after the snapshot 2023-10-30T12:48:01.114 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/33 file after the snapshot 2023-10-30T12:48:01.115 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/34 file after the snapshot 2023-10-30T12:48:01.116 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/35 file after the snapshot 2023-10-30T12:48:01.116 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/36 file after the snapshot 2023-10-30T12:48:01.117 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/37 file after the snapshot 2023-10-30T12:48:01.118 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/38 file after the snapshot 2023-10-30T12:48:01.119 INFO:tasks.workunit.client.0.smithi008.stdout: create 101/39 file after the snapshot 2023-10-30T12:48:01.119 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/.snap/snap-subdir-test 2023-10-30T12:48:01.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/30 file after the snapshot 2023-10-30T12:48:01.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/31 file after the snapshot 2023-10-30T12:48:01.139 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/32 file after the snapshot 2023-10-30T12:48:01.140 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/33 file after the snapshot 2023-10-30T12:48:01.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/34 file after the snapshot 2023-10-30T12:48:01.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/35 file after the snapshot 2023-10-30T12:48:01.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/36 file after the snapshot 2023-10-30T12:48:01.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/37 file after the snapshot 2023-10-30T12:48:01.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/38 file after the snapshot 2023-10-30T12:48:01.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 102/39 file after the snapshot 2023-10-30T12:48:01.145 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/.snap/snap-subdir-test 2023-10-30T12:48:01.167 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/30 file after the snapshot 2023-10-30T12:48:01.206 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/31 file after the snapshot 2023-10-30T12:48:01.206 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/32 file after the snapshot 2023-10-30T12:48:01.206 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/33 file after the snapshot 2023-10-30T12:48:01.206 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/34 file after the snapshot 2023-10-30T12:48:01.207 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/35 file after the snapshot 2023-10-30T12:48:01.207 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/36 file after the snapshot 2023-10-30T12:48:01.207 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/37 file after the snapshot 2023-10-30T12:48:01.207 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/38 file after the snapshot 2023-10-30T12:48:01.207 INFO:tasks.workunit.client.0.smithi008.stdout: create 103/39 file after the snapshot 2023-10-30T12:48:01.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/.snap/snap-subdir-test 2023-10-30T12:48:01.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/30 file after the snapshot 2023-10-30T12:48:01.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/31 file after the snapshot 2023-10-30T12:48:01.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/32 file after the snapshot 2023-10-30T12:48:01.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/33 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/34 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/35 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/36 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/37 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/38 file after the snapshot 2023-10-30T12:48:01.209 INFO:tasks.workunit.client.0.smithi008.stdout: create 104/39 file after the snapshot 2023-10-30T12:48:01.210 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/.snap/snap-subdir-test 2023-10-30T12:48:01.222 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/30 file after the snapshot 2023-10-30T12:48:01.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/31 file after the snapshot 2023-10-30T12:48:01.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/32 file after the snapshot 2023-10-30T12:48:01.224 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/33 file after the snapshot 2023-10-30T12:48:01.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/34 file after the snapshot 2023-10-30T12:48:01.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/35 file after the snapshot 2023-10-30T12:48:01.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/36 file after the snapshot 2023-10-30T12:48:01.227 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/37 file after the snapshot 2023-10-30T12:48:01.227 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/38 file after the snapshot 2023-10-30T12:48:01.228 INFO:tasks.workunit.client.0.smithi008.stdout: create 105/39 file after the snapshot 2023-10-30T12:48:01.229 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/.snap/snap-subdir-test 2023-10-30T12:48:01.250 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/30 file after the snapshot 2023-10-30T12:48:01.251 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/31 file after the snapshot 2023-10-30T12:48:01.252 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/32 file after the snapshot 2023-10-30T12:48:01.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/33 file after the snapshot 2023-10-30T12:48:01.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/34 file after the snapshot 2023-10-30T12:48:01.254 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/35 file after the snapshot 2023-10-30T12:48:01.255 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/36 file after the snapshot 2023-10-30T12:48:01.255 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/37 file after the snapshot 2023-10-30T12:48:01.256 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/38 file after the snapshot 2023-10-30T12:48:01.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 106/39 file after the snapshot 2023-10-30T12:48:01.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/.snap/snap-subdir-test 2023-10-30T12:48:01.279 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/30 file after the snapshot 2023-10-30T12:48:01.280 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/31 file after the snapshot 2023-10-30T12:48:01.280 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/32 file after the snapshot 2023-10-30T12:48:01.281 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/33 file after the snapshot 2023-10-30T12:48:01.282 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/34 file after the snapshot 2023-10-30T12:48:01.282 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/35 file after the snapshot 2023-10-30T12:48:01.283 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/36 file after the snapshot 2023-10-30T12:48:01.284 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/37 file after the snapshot 2023-10-30T12:48:01.285 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/38 file after the snapshot 2023-10-30T12:48:01.285 INFO:tasks.workunit.client.0.smithi008.stdout: create 107/39 file after the snapshot 2023-10-30T12:48:01.286 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/.snap/snap-subdir-test 2023-10-30T12:48:01.304 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/30 file after the snapshot 2023-10-30T12:48:01.306 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/31 file after the snapshot 2023-10-30T12:48:01.306 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/32 file after the snapshot 2023-10-30T12:48:01.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/33 file after the snapshot 2023-10-30T12:48:01.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/34 file after the snapshot 2023-10-30T12:48:01.308 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/35 file after the snapshot 2023-10-30T12:48:01.309 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/36 file after the snapshot 2023-10-30T12:48:01.309 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/37 file after the snapshot 2023-10-30T12:48:01.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/38 file after the snapshot 2023-10-30T12:48:01.311 INFO:tasks.workunit.client.0.smithi008.stdout: create 108/39 file after the snapshot 2023-10-30T12:48:01.311 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/.snap/snap-subdir-test 2023-10-30T12:48:01.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/30 file after the snapshot 2023-10-30T12:48:01.336 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/31 file after the snapshot 2023-10-30T12:48:01.336 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/32 file after the snapshot 2023-10-30T12:48:01.337 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/33 file after the snapshot 2023-10-30T12:48:01.338 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/34 file after the snapshot 2023-10-30T12:48:01.338 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/35 file after the snapshot 2023-10-30T12:48:01.339 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/36 file after the snapshot 2023-10-30T12:48:01.340 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/37 file after the snapshot 2023-10-30T12:48:01.341 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/38 file after the snapshot 2023-10-30T12:48:01.341 INFO:tasks.workunit.client.0.smithi008.stdout: create 109/39 file after the snapshot 2023-10-30T12:48:01.342 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/.snap/snap-subdir-test 2023-10-30T12:48:01.364 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/30 file after the snapshot 2023-10-30T12:48:01.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/31 file after the snapshot 2023-10-30T12:48:01.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/32 file after the snapshot 2023-10-30T12:48:01.366 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/33 file after the snapshot 2023-10-30T12:48:01.367 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/34 file after the snapshot 2023-10-30T12:48:01.367 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/35 file after the snapshot 2023-10-30T12:48:01.368 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/36 file after the snapshot 2023-10-30T12:48:01.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/37 file after the snapshot 2023-10-30T12:48:01.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/38 file after the snapshot 2023-10-30T12:48:01.370 INFO:tasks.workunit.client.0.smithi008.stdout: create 110/39 file after the snapshot 2023-10-30T12:48:01.371 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/.snap/snap-subdir-test 2023-10-30T12:48:01.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/30 file after the snapshot 2023-10-30T12:48:01.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/31 file after the snapshot 2023-10-30T12:48:01.393 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/32 file after the snapshot 2023-10-30T12:48:01.393 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/33 file after the snapshot 2023-10-30T12:48:01.394 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/34 file after the snapshot 2023-10-30T12:48:01.395 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/35 file after the snapshot 2023-10-30T12:48:01.396 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/36 file after the snapshot 2023-10-30T12:48:01.396 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/37 file after the snapshot 2023-10-30T12:48:01.397 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/38 file after the snapshot 2023-10-30T12:48:01.398 INFO:tasks.workunit.client.0.smithi008.stdout: create 111/39 file after the snapshot 2023-10-30T12:48:01.399 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/.snap/snap-subdir-test 2023-10-30T12:48:01.417 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/30 file after the snapshot 2023-10-30T12:48:01.418 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/31 file after the snapshot 2023-10-30T12:48:01.419 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/32 file after the snapshot 2023-10-30T12:48:01.419 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/33 file after the snapshot 2023-10-30T12:48:01.420 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/34 file after the snapshot 2023-10-30T12:48:01.421 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/35 file after the snapshot 2023-10-30T12:48:01.421 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/36 file after the snapshot 2023-10-30T12:48:01.422 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/37 file after the snapshot 2023-10-30T12:48:01.423 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/38 file after the snapshot 2023-10-30T12:48:01.424 INFO:tasks.workunit.client.0.smithi008.stdout: create 112/39 file after the snapshot 2023-10-30T12:48:01.424 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/.snap/snap-subdir-test 2023-10-30T12:48:01.447 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/30 file after the snapshot 2023-10-30T12:48:01.448 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/31 file after the snapshot 2023-10-30T12:48:01.449 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/32 file after the snapshot 2023-10-30T12:48:01.449 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/33 file after the snapshot 2023-10-30T12:48:01.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/34 file after the snapshot 2023-10-30T12:48:01.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/35 file after the snapshot 2023-10-30T12:48:01.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/36 file after the snapshot 2023-10-30T12:48:01.452 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/37 file after the snapshot 2023-10-30T12:48:01.453 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/38 file after the snapshot 2023-10-30T12:48:01.453 INFO:tasks.workunit.client.0.smithi008.stdout: create 113/39 file after the snapshot 2023-10-30T12:48:01.454 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/.snap/snap-subdir-test 2023-10-30T12:48:01.477 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/30 file after the snapshot 2023-10-30T12:48:01.478 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/31 file after the snapshot 2023-10-30T12:48:01.478 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/32 file after the snapshot 2023-10-30T12:48:01.479 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/33 file after the snapshot 2023-10-30T12:48:01.480 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/34 file after the snapshot 2023-10-30T12:48:01.480 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/35 file after the snapshot 2023-10-30T12:48:01.481 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/36 file after the snapshot 2023-10-30T12:48:01.482 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/37 file after the snapshot 2023-10-30T12:48:01.483 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/38 file after the snapshot 2023-10-30T12:48:01.483 INFO:tasks.workunit.client.0.smithi008.stdout: create 114/39 file after the snapshot 2023-10-30T12:48:01.484 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/.snap/snap-subdir-test 2023-10-30T12:48:01.505 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/30 file after the snapshot 2023-10-30T12:48:01.506 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/31 file after the snapshot 2023-10-30T12:48:01.506 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/32 file after the snapshot 2023-10-30T12:48:01.507 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/33 file after the snapshot 2023-10-30T12:48:01.508 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/34 file after the snapshot 2023-10-30T12:48:01.508 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/35 file after the snapshot 2023-10-30T12:48:01.510 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/36 file after the snapshot 2023-10-30T12:48:01.510 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/37 file after the snapshot 2023-10-30T12:48:01.511 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/38 file after the snapshot 2023-10-30T12:48:01.511 INFO:tasks.workunit.client.0.smithi008.stdout: create 115/39 file after the snapshot 2023-10-30T12:48:01.512 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/.snap/snap-subdir-test 2023-10-30T12:48:01.533 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/30 file after the snapshot 2023-10-30T12:48:01.535 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/31 file after the snapshot 2023-10-30T12:48:01.536 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/32 file after the snapshot 2023-10-30T12:48:01.537 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/33 file after the snapshot 2023-10-30T12:48:01.538 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/34 file after the snapshot 2023-10-30T12:48:01.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/35 file after the snapshot 2023-10-30T12:48:01.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/36 file after the snapshot 2023-10-30T12:48:01.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/37 file after the snapshot 2023-10-30T12:48:01.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/38 file after the snapshot 2023-10-30T12:48:01.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 116/39 file after the snapshot 2023-10-30T12:48:01.544 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/.snap/snap-subdir-test 2023-10-30T12:48:01.564 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/30 file after the snapshot 2023-10-30T12:48:01.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/31 file after the snapshot 2023-10-30T12:48:01.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/32 file after the snapshot 2023-10-30T12:48:01.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/33 file after the snapshot 2023-10-30T12:48:01.567 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/34 file after the snapshot 2023-10-30T12:48:01.567 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/35 file after the snapshot 2023-10-30T12:48:01.568 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/36 file after the snapshot 2023-10-30T12:48:01.569 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/37 file after the snapshot 2023-10-30T12:48:01.570 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/38 file after the snapshot 2023-10-30T12:48:01.570 INFO:tasks.workunit.client.0.smithi008.stdout: create 117/39 file after the snapshot 2023-10-30T12:48:01.571 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/.snap/snap-subdir-test 2023-10-30T12:48:01.591 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/30 file after the snapshot 2023-10-30T12:48:01.592 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/31 file after the snapshot 2023-10-30T12:48:01.593 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/32 file after the snapshot 2023-10-30T12:48:01.593 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/33 file after the snapshot 2023-10-30T12:48:01.594 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/34 file after the snapshot 2023-10-30T12:48:01.595 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/35 file after the snapshot 2023-10-30T12:48:01.595 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/36 file after the snapshot 2023-10-30T12:48:01.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/37 file after the snapshot 2023-10-30T12:48:01.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/38 file after the snapshot 2023-10-30T12:48:01.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 118/39 file after the snapshot 2023-10-30T12:48:01.598 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/.snap/snap-subdir-test 2023-10-30T12:48:01.619 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/30 file after the snapshot 2023-10-30T12:48:01.620 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/31 file after the snapshot 2023-10-30T12:48:01.621 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/32 file after the snapshot 2023-10-30T12:48:01.621 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/33 file after the snapshot 2023-10-30T12:48:01.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/34 file after the snapshot 2023-10-30T12:48:01.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/35 file after the snapshot 2023-10-30T12:48:01.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/36 file after the snapshot 2023-10-30T12:48:01.624 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/37 file after the snapshot 2023-10-30T12:48:01.625 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/38 file after the snapshot 2023-10-30T12:48:01.626 INFO:tasks.workunit.client.0.smithi008.stdout: create 119/39 file after the snapshot 2023-10-30T12:48:01.626 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/.snap/snap-subdir-test 2023-10-30T12:48:01.648 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/30 file after the snapshot 2023-10-30T12:48:01.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/31 file after the snapshot 2023-10-30T12:48:01.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/32 file after the snapshot 2023-10-30T12:48:01.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/33 file after the snapshot 2023-10-30T12:48:01.651 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/34 file after the snapshot 2023-10-30T12:48:01.652 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/35 file after the snapshot 2023-10-30T12:48:01.653 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/36 file after the snapshot 2023-10-30T12:48:01.653 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/37 file after the snapshot 2023-10-30T12:48:01.654 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/38 file after the snapshot 2023-10-30T12:48:01.655 INFO:tasks.workunit.client.0.smithi008.stdout: create 120/39 file after the snapshot 2023-10-30T12:48:01.655 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/.snap/snap-subdir-test 2023-10-30T12:48:01.677 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/30 file after the snapshot 2023-10-30T12:48:01.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/31 file after the snapshot 2023-10-30T12:48:01.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/32 file after the snapshot 2023-10-30T12:48:01.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/33 file after the snapshot 2023-10-30T12:48:01.682 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/34 file after the snapshot 2023-10-30T12:48:01.683 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/35 file after the snapshot 2023-10-30T12:48:01.684 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/36 file after the snapshot 2023-10-30T12:48:01.685 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/37 file after the snapshot 2023-10-30T12:48:01.686 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/38 file after the snapshot 2023-10-30T12:48:01.687 INFO:tasks.workunit.client.0.smithi008.stdout: create 121/39 file after the snapshot 2023-10-30T12:48:01.688 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/.snap/snap-subdir-test 2023-10-30T12:48:01.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/30 file after the snapshot 2023-10-30T12:48:01.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/31 file after the snapshot 2023-10-30T12:48:01.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/32 file after the snapshot 2023-10-30T12:48:01.712 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/33 file after the snapshot 2023-10-30T12:48:01.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/34 file after the snapshot 2023-10-30T12:48:01.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/35 file after the snapshot 2023-10-30T12:48:01.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/36 file after the snapshot 2023-10-30T12:48:01.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/37 file after the snapshot 2023-10-30T12:48:01.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/38 file after the snapshot 2023-10-30T12:48:01.716 INFO:tasks.workunit.client.0.smithi008.stdout: create 122/39 file after the snapshot 2023-10-30T12:48:01.717 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/.snap/snap-subdir-test 2023-10-30T12:48:01.737 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/30 file after the snapshot 2023-10-30T12:48:01.738 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/31 file after the snapshot 2023-10-30T12:48:01.739 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/32 file after the snapshot 2023-10-30T12:48:01.739 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/33 file after the snapshot 2023-10-30T12:48:01.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/34 file after the snapshot 2023-10-30T12:48:01.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/35 file after the snapshot 2023-10-30T12:48:01.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/36 file after the snapshot 2023-10-30T12:48:01.742 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/37 file after the snapshot 2023-10-30T12:48:01.743 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/38 file after the snapshot 2023-10-30T12:48:01.743 INFO:tasks.workunit.client.0.smithi008.stdout: create 123/39 file after the snapshot 2023-10-30T12:48:01.744 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/.snap/snap-subdir-test 2023-10-30T12:48:01.763 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/30 file after the snapshot 2023-10-30T12:48:01.764 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/31 file after the snapshot 2023-10-30T12:48:01.765 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/32 file after the snapshot 2023-10-30T12:48:01.766 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/33 file after the snapshot 2023-10-30T12:48:01.767 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/34 file after the snapshot 2023-10-30T12:48:01.768 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/35 file after the snapshot 2023-10-30T12:48:01.769 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/36 file after the snapshot 2023-10-30T12:48:01.770 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/37 file after the snapshot 2023-10-30T12:48:01.774 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/38 file after the snapshot 2023-10-30T12:48:01.775 INFO:tasks.workunit.client.0.smithi008.stdout: create 124/39 file after the snapshot 2023-10-30T12:48:01.775 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/.snap/snap-subdir-test 2023-10-30T12:48:01.795 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/30 file after the snapshot 2023-10-30T12:48:01.796 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/31 file after the snapshot 2023-10-30T12:48:01.797 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/32 file after the snapshot 2023-10-30T12:48:01.798 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/33 file after the snapshot 2023-10-30T12:48:01.799 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/34 file after the snapshot 2023-10-30T12:48:01.800 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/35 file after the snapshot 2023-10-30T12:48:01.801 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/36 file after the snapshot 2023-10-30T12:48:01.802 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/37 file after the snapshot 2023-10-30T12:48:01.803 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/38 file after the snapshot 2023-10-30T12:48:01.804 INFO:tasks.workunit.client.0.smithi008.stdout: create 125/39 file after the snapshot 2023-10-30T12:48:01.805 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/.snap/snap-subdir-test 2023-10-30T12:48:01.825 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/30 file after the snapshot 2023-10-30T12:48:01.826 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/31 file after the snapshot 2023-10-30T12:48:01.826 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/32 file after the snapshot 2023-10-30T12:48:01.827 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/33 file after the snapshot 2023-10-30T12:48:01.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/34 file after the snapshot 2023-10-30T12:48:01.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/35 file after the snapshot 2023-10-30T12:48:01.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/36 file after the snapshot 2023-10-30T12:48:01.830 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/37 file after the snapshot 2023-10-30T12:48:01.830 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/38 file after the snapshot 2023-10-30T12:48:01.831 INFO:tasks.workunit.client.0.smithi008.stdout: create 126/39 file after the snapshot 2023-10-30T12:48:01.832 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/.snap/snap-subdir-test 2023-10-30T12:48:02.180 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/30 file after the snapshot 2023-10-30T12:48:02.181 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/31 file after the snapshot 2023-10-30T12:48:02.182 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/32 file after the snapshot 2023-10-30T12:48:02.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/33 file after the snapshot 2023-10-30T12:48:02.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/34 file after the snapshot 2023-10-30T12:48:02.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/35 file after the snapshot 2023-10-30T12:48:02.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/36 file after the snapshot 2023-10-30T12:48:02.186 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/37 file after the snapshot 2023-10-30T12:48:02.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/38 file after the snapshot 2023-10-30T12:48:02.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 127/39 file after the snapshot 2023-10-30T12:48:02.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/.snap/snap-subdir-test 2023-10-30T12:48:02.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/30 file after the snapshot 2023-10-30T12:48:02.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/31 file after the snapshot 2023-10-30T12:48:02.213 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/32 file after the snapshot 2023-10-30T12:48:02.214 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/33 file after the snapshot 2023-10-30T12:48:02.215 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/34 file after the snapshot 2023-10-30T12:48:02.216 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/35 file after the snapshot 2023-10-30T12:48:02.218 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/36 file after the snapshot 2023-10-30T12:48:02.219 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/37 file after the snapshot 2023-10-30T12:48:02.220 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/38 file after the snapshot 2023-10-30T12:48:02.221 INFO:tasks.workunit.client.0.smithi008.stdout: create 128/39 file after the snapshot 2023-10-30T12:48:02.222 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/.snap/snap-subdir-test 2023-10-30T12:48:02.242 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/30 file after the snapshot 2023-10-30T12:48:02.243 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/31 file after the snapshot 2023-10-30T12:48:02.244 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/32 file after the snapshot 2023-10-30T12:48:02.244 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/33 file after the snapshot 2023-10-30T12:48:02.245 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/34 file after the snapshot 2023-10-30T12:48:02.246 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/35 file after the snapshot 2023-10-30T12:48:02.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/36 file after the snapshot 2023-10-30T12:48:02.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/37 file after the snapshot 2023-10-30T12:48:02.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/38 file after the snapshot 2023-10-30T12:48:02.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 129/39 file after the snapshot 2023-10-30T12:48:02.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/.snap/snap-subdir-test 2023-10-30T12:48:02.273 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/30 file after the snapshot 2023-10-30T12:48:02.274 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/31 file after the snapshot 2023-10-30T12:48:02.275 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/32 file after the snapshot 2023-10-30T12:48:02.275 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/33 file after the snapshot 2023-10-30T12:48:02.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/34 file after the snapshot 2023-10-30T12:48:02.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/35 file after the snapshot 2023-10-30T12:48:02.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/36 file after the snapshot 2023-10-30T12:48:02.278 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/37 file after the snapshot 2023-10-30T12:48:02.279 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/38 file after the snapshot 2023-10-30T12:48:02.280 INFO:tasks.workunit.client.0.smithi008.stdout: create 130/39 file after the snapshot 2023-10-30T12:48:02.280 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/.snap/snap-subdir-test 2023-10-30T12:48:02.300 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/30 file after the snapshot 2023-10-30T12:48:02.301 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/31 file after the snapshot 2023-10-30T12:48:02.301 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/32 file after the snapshot 2023-10-30T12:48:02.302 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/33 file after the snapshot 2023-10-30T12:48:02.303 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/34 file after the snapshot 2023-10-30T12:48:02.304 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/35 file after the snapshot 2023-10-30T12:48:02.305 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/36 file after the snapshot 2023-10-30T12:48:02.305 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/37 file after the snapshot 2023-10-30T12:48:02.306 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/38 file after the snapshot 2023-10-30T12:48:02.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 131/39 file after the snapshot 2023-10-30T12:48:02.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/.snap/snap-subdir-test 2023-10-30T12:48:02.325 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/30 file after the snapshot 2023-10-30T12:48:02.326 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/31 file after the snapshot 2023-10-30T12:48:02.326 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/32 file after the snapshot 2023-10-30T12:48:02.327 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/33 file after the snapshot 2023-10-30T12:48:02.328 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/34 file after the snapshot 2023-10-30T12:48:02.328 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/35 file after the snapshot 2023-10-30T12:48:02.329 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/36 file after the snapshot 2023-10-30T12:48:02.330 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/37 file after the snapshot 2023-10-30T12:48:02.331 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/38 file after the snapshot 2023-10-30T12:48:02.331 INFO:tasks.workunit.client.0.smithi008.stdout: create 132/39 file after the snapshot 2023-10-30T12:48:02.332 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/.snap/snap-subdir-test 2023-10-30T12:48:02.351 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/30 file after the snapshot 2023-10-30T12:48:02.352 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/31 file after the snapshot 2023-10-30T12:48:02.353 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/32 file after the snapshot 2023-10-30T12:48:02.353 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/33 file after the snapshot 2023-10-30T12:48:02.354 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/34 file after the snapshot 2023-10-30T12:48:02.355 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/35 file after the snapshot 2023-10-30T12:48:02.356 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/36 file after the snapshot 2023-10-30T12:48:02.356 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/37 file after the snapshot 2023-10-30T12:48:02.357 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/38 file after the snapshot 2023-10-30T12:48:02.358 INFO:tasks.workunit.client.0.smithi008.stdout: create 133/39 file after the snapshot 2023-10-30T12:48:02.358 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/.snap/snap-subdir-test 2023-10-30T12:48:02.381 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/30 file after the snapshot 2023-10-30T12:48:02.382 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/31 file after the snapshot 2023-10-30T12:48:02.383 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/32 file after the snapshot 2023-10-30T12:48:02.383 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/33 file after the snapshot 2023-10-30T12:48:02.384 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/34 file after the snapshot 2023-10-30T12:48:02.385 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/35 file after the snapshot 2023-10-30T12:48:02.386 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/36 file after the snapshot 2023-10-30T12:48:02.386 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/37 file after the snapshot 2023-10-30T12:48:02.387 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/38 file after the snapshot 2023-10-30T12:48:02.388 INFO:tasks.workunit.client.0.smithi008.stdout: create 134/39 file after the snapshot 2023-10-30T12:48:02.388 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/.snap/snap-subdir-test 2023-10-30T12:48:03.394 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/30 file after the snapshot 2023-10-30T12:48:03.395 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/31 file after the snapshot 2023-10-30T12:48:03.396 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/32 file after the snapshot 2023-10-30T12:48:03.397 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/33 file after the snapshot 2023-10-30T12:48:03.398 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/34 file after the snapshot 2023-10-30T12:48:03.399 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/35 file after the snapshot 2023-10-30T12:48:03.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/36 file after the snapshot 2023-10-30T12:48:03.401 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/37 file after the snapshot 2023-10-30T12:48:03.401 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/38 file after the snapshot 2023-10-30T12:48:03.402 INFO:tasks.workunit.client.0.smithi008.stdout: create 135/39 file after the snapshot 2023-10-30T12:48:03.403 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/.snap/snap-subdir-test 2023-10-30T12:48:03.425 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/30 file after the snapshot 2023-10-30T12:48:03.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/31 file after the snapshot 2023-10-30T12:48:03.427 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/32 file after the snapshot 2023-10-30T12:48:03.427 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/33 file after the snapshot 2023-10-30T12:48:03.428 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/34 file after the snapshot 2023-10-30T12:48:03.429 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/35 file after the snapshot 2023-10-30T12:48:03.430 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/36 file after the snapshot 2023-10-30T12:48:03.431 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/37 file after the snapshot 2023-10-30T12:48:03.432 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/38 file after the snapshot 2023-10-30T12:48:03.432 INFO:tasks.workunit.client.0.smithi008.stdout: create 136/39 file after the snapshot 2023-10-30T12:48:03.433 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/.snap/snap-subdir-test 2023-10-30T12:48:03.457 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/30 file after the snapshot 2023-10-30T12:48:03.458 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/31 file after the snapshot 2023-10-30T12:48:03.459 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/32 file after the snapshot 2023-10-30T12:48:03.460 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/33 file after the snapshot 2023-10-30T12:48:03.461 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/34 file after the snapshot 2023-10-30T12:48:03.462 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/35 file after the snapshot 2023-10-30T12:48:03.462 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/36 file after the snapshot 2023-10-30T12:48:03.463 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/37 file after the snapshot 2023-10-30T12:48:03.464 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/38 file after the snapshot 2023-10-30T12:48:03.465 INFO:tasks.workunit.client.0.smithi008.stdout: create 137/39 file after the snapshot 2023-10-30T12:48:03.466 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/.snap/snap-subdir-test 2023-10-30T12:48:03.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/30 file after the snapshot 2023-10-30T12:48:03.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/31 file after the snapshot 2023-10-30T12:48:03.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/32 file after the snapshot 2023-10-30T12:48:03.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/33 file after the snapshot 2023-10-30T12:48:03.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/34 file after the snapshot 2023-10-30T12:48:03.742 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/35 file after the snapshot 2023-10-30T12:48:03.743 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/36 file after the snapshot 2023-10-30T12:48:03.744 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/37 file after the snapshot 2023-10-30T12:48:03.745 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/38 file after the snapshot 2023-10-30T12:48:03.745 INFO:tasks.workunit.client.0.smithi008.stdout: create 138/39 file after the snapshot 2023-10-30T12:48:03.746 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/.snap/snap-subdir-test 2023-10-30T12:48:03.772 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/30 file after the snapshot 2023-10-30T12:48:04.024 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/31 file after the snapshot 2023-10-30T12:48:04.025 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/32 file after the snapshot 2023-10-30T12:48:04.026 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/33 file after the snapshot 2023-10-30T12:48:04.027 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/34 file after the snapshot 2023-10-30T12:48:04.028 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/35 file after the snapshot 2023-10-30T12:48:04.029 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/36 file after the snapshot 2023-10-30T12:48:04.029 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/37 file after the snapshot 2023-10-30T12:48:04.030 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/38 file after the snapshot 2023-10-30T12:48:04.031 INFO:tasks.workunit.client.0.smithi008.stdout: create 139/39 file after the snapshot 2023-10-30T12:48:04.032 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/.snap/snap-subdir-test 2023-10-30T12:48:04.056 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/30 file after the snapshot 2023-10-30T12:48:04.058 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/31 file after the snapshot 2023-10-30T12:48:04.058 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/32 file after the snapshot 2023-10-30T12:48:04.059 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/33 file after the snapshot 2023-10-30T12:48:04.060 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/34 file after the snapshot 2023-10-30T12:48:04.061 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/35 file after the snapshot 2023-10-30T12:48:04.062 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/36 file after the snapshot 2023-10-30T12:48:04.062 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/37 file after the snapshot 2023-10-30T12:48:04.063 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/38 file after the snapshot 2023-10-30T12:48:04.064 INFO:tasks.workunit.client.0.smithi008.stdout: create 140/39 file after the snapshot 2023-10-30T12:48:04.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/.snap/snap-subdir-test 2023-10-30T12:48:04.088 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/30 file after the snapshot 2023-10-30T12:48:04.089 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/31 file after the snapshot 2023-10-30T12:48:04.089 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/32 file after the snapshot 2023-10-30T12:48:04.090 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/33 file after the snapshot 2023-10-30T12:48:04.091 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/34 file after the snapshot 2023-10-30T12:48:04.092 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/35 file after the snapshot 2023-10-30T12:48:04.093 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/36 file after the snapshot 2023-10-30T12:48:04.093 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/37 file after the snapshot 2023-10-30T12:48:04.094 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/38 file after the snapshot 2023-10-30T12:48:04.095 INFO:tasks.workunit.client.0.smithi008.stdout: create 141/39 file after the snapshot 2023-10-30T12:48:04.096 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/.snap/snap-subdir-test 2023-10-30T12:48:04.137 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/30 file after the snapshot 2023-10-30T12:48:04.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/31 file after the snapshot 2023-10-30T12:48:04.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/32 file after the snapshot 2023-10-30T12:48:04.139 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/33 file after the snapshot 2023-10-30T12:48:04.140 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/34 file after the snapshot 2023-10-30T12:48:04.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/35 file after the snapshot 2023-10-30T12:48:04.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/36 file after the snapshot 2023-10-30T12:48:04.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/37 file after the snapshot 2023-10-30T12:48:04.145 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/38 file after the snapshot 2023-10-30T12:48:04.146 INFO:tasks.workunit.client.0.smithi008.stdout: create 142/39 file after the snapshot 2023-10-30T12:48:04.148 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/.snap/snap-subdir-test 2023-10-30T12:48:04.935 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/30 file after the snapshot 2023-10-30T12:48:04.936 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/31 file after the snapshot 2023-10-30T12:48:04.937 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/32 file after the snapshot 2023-10-30T12:48:04.937 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/33 file after the snapshot 2023-10-30T12:48:04.938 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/34 file after the snapshot 2023-10-30T12:48:04.939 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/35 file after the snapshot 2023-10-30T12:48:04.940 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/36 file after the snapshot 2023-10-30T12:48:04.940 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/37 file after the snapshot 2023-10-30T12:48:04.941 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/38 file after the snapshot 2023-10-30T12:48:04.941 INFO:tasks.workunit.client.0.smithi008.stdout: create 143/39 file after the snapshot 2023-10-30T12:48:04.942 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/.snap/snap-subdir-test 2023-10-30T12:48:04.994 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/30 file after the snapshot 2023-10-30T12:48:04.995 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/31 file after the snapshot 2023-10-30T12:48:04.996 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/32 file after the snapshot 2023-10-30T12:48:04.996 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/33 file after the snapshot 2023-10-30T12:48:04.997 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/34 file after the snapshot 2023-10-30T12:48:04.998 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/35 file after the snapshot 2023-10-30T12:48:04.998 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/36 file after the snapshot 2023-10-30T12:48:04.999 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/37 file after the snapshot 2023-10-30T12:48:05.000 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/38 file after the snapshot 2023-10-30T12:48:05.000 INFO:tasks.workunit.client.0.smithi008.stdout: create 144/39 file after the snapshot 2023-10-30T12:48:05.001 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/.snap/snap-subdir-test 2023-10-30T12:48:05.408 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/30 file after the snapshot 2023-10-30T12:48:05.409 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/31 file after the snapshot 2023-10-30T12:48:05.409 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/32 file after the snapshot 2023-10-30T12:48:05.410 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/33 file after the snapshot 2023-10-30T12:48:05.411 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/34 file after the snapshot 2023-10-30T12:48:05.412 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/35 file after the snapshot 2023-10-30T12:48:05.413 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/36 file after the snapshot 2023-10-30T12:48:05.413 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/37 file after the snapshot 2023-10-30T12:48:05.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/38 file after the snapshot 2023-10-30T12:48:05.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 145/39 file after the snapshot 2023-10-30T12:48:05.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/.snap/snap-subdir-test 2023-10-30T12:48:05.436 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/30 file after the snapshot 2023-10-30T12:48:05.437 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/31 file after the snapshot 2023-10-30T12:48:05.437 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/32 file after the snapshot 2023-10-30T12:48:05.438 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/33 file after the snapshot 2023-10-30T12:48:05.439 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/34 file after the snapshot 2023-10-30T12:48:05.440 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/35 file after the snapshot 2023-10-30T12:48:05.440 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/36 file after the snapshot 2023-10-30T12:48:05.441 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/37 file after the snapshot 2023-10-30T12:48:05.442 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/38 file after the snapshot 2023-10-30T12:48:05.443 INFO:tasks.workunit.client.0.smithi008.stdout: create 146/39 file after the snapshot 2023-10-30T12:48:05.443 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/.snap/snap-subdir-test 2023-10-30T12:48:05.461 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/30 file after the snapshot 2023-10-30T12:48:05.462 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/31 file after the snapshot 2023-10-30T12:48:05.462 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/32 file after the snapshot 2023-10-30T12:48:05.463 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/33 file after the snapshot 2023-10-30T12:48:05.464 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/34 file after the snapshot 2023-10-30T12:48:05.465 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/35 file after the snapshot 2023-10-30T12:48:05.465 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/36 file after the snapshot 2023-10-30T12:48:05.466 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/37 file after the snapshot 2023-10-30T12:48:05.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/38 file after the snapshot 2023-10-30T12:48:05.468 INFO:tasks.workunit.client.0.smithi008.stdout: create 147/39 file after the snapshot 2023-10-30T12:48:05.468 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/.snap/snap-subdir-test 2023-10-30T12:48:05.487 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/30 file after the snapshot 2023-10-30T12:48:05.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/31 file after the snapshot 2023-10-30T12:48:05.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/32 file after the snapshot 2023-10-30T12:48:05.490 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/33 file after the snapshot 2023-10-30T12:48:05.491 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/34 file after the snapshot 2023-10-30T12:48:05.491 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/35 file after the snapshot 2023-10-30T12:48:05.492 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/36 file after the snapshot 2023-10-30T12:48:05.493 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/37 file after the snapshot 2023-10-30T12:48:05.493 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/38 file after the snapshot 2023-10-30T12:48:05.494 INFO:tasks.workunit.client.0.smithi008.stdout: create 148/39 file after the snapshot 2023-10-30T12:48:05.495 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/.snap/snap-subdir-test 2023-10-30T12:48:05.512 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/30 file after the snapshot 2023-10-30T12:48:05.513 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/31 file after the snapshot 2023-10-30T12:48:05.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/32 file after the snapshot 2023-10-30T12:48:05.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/33 file after the snapshot 2023-10-30T12:48:05.515 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/34 file after the snapshot 2023-10-30T12:48:05.516 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/35 file after the snapshot 2023-10-30T12:48:05.516 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/36 file after the snapshot 2023-10-30T12:48:05.517 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/37 file after the snapshot 2023-10-30T12:48:05.518 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/38 file after the snapshot 2023-10-30T12:48:05.518 INFO:tasks.workunit.client.0.smithi008.stdout: create 149/39 file after the snapshot 2023-10-30T12:48:05.519 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/.snap/snap-subdir-test 2023-10-30T12:48:05.538 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/30 file after the snapshot 2023-10-30T12:48:05.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/31 file after the snapshot 2023-10-30T12:48:05.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/32 file after the snapshot 2023-10-30T12:48:05.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/33 file after the snapshot 2023-10-30T12:48:05.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/34 file after the snapshot 2023-10-30T12:48:05.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/35 file after the snapshot 2023-10-30T12:48:05.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/36 file after the snapshot 2023-10-30T12:48:05.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/37 file after the snapshot 2023-10-30T12:48:05.544 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/38 file after the snapshot 2023-10-30T12:48:05.545 INFO:tasks.workunit.client.0.smithi008.stdout: create 150/39 file after the snapshot 2023-10-30T12:48:05.545 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/.snap/snap-subdir-test 2023-10-30T12:48:05.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/30 file after the snapshot 2023-10-30T12:48:05.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/31 file after the snapshot 2023-10-30T12:48:05.567 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/32 file after the snapshot 2023-10-30T12:48:05.569 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/33 file after the snapshot 2023-10-30T12:48:05.569 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/34 file after the snapshot 2023-10-30T12:48:05.570 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/35 file after the snapshot 2023-10-30T12:48:05.571 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/36 file after the snapshot 2023-10-30T12:48:05.572 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/37 file after the snapshot 2023-10-30T12:48:05.573 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/38 file after the snapshot 2023-10-30T12:48:05.574 INFO:tasks.workunit.client.0.smithi008.stdout: create 151/39 file after the snapshot 2023-10-30T12:48:05.575 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/.snap/snap-subdir-test 2023-10-30T12:48:05.595 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/30 file after the snapshot 2023-10-30T12:48:05.595 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/31 file after the snapshot 2023-10-30T12:48:05.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/32 file after the snapshot 2023-10-30T12:48:05.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/33 file after the snapshot 2023-10-30T12:48:05.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/34 file after the snapshot 2023-10-30T12:48:05.598 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/35 file after the snapshot 2023-10-30T12:48:05.599 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/36 file after the snapshot 2023-10-30T12:48:05.600 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/37 file after the snapshot 2023-10-30T12:48:05.600 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/38 file after the snapshot 2023-10-30T12:48:05.601 INFO:tasks.workunit.client.0.smithi008.stdout: create 152/39 file after the snapshot 2023-10-30T12:48:05.601 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/.snap/snap-subdir-test 2023-10-30T12:48:05.621 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/30 file after the snapshot 2023-10-30T12:48:05.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/31 file after the snapshot 2023-10-30T12:48:05.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/32 file after the snapshot 2023-10-30T12:48:05.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/33 file after the snapshot 2023-10-30T12:48:05.624 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/34 file after the snapshot 2023-10-30T12:48:05.625 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/35 file after the snapshot 2023-10-30T12:48:05.626 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/36 file after the snapshot 2023-10-30T12:48:05.626 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/37 file after the snapshot 2023-10-30T12:48:05.627 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/38 file after the snapshot 2023-10-30T12:48:05.628 INFO:tasks.workunit.client.0.smithi008.stdout: create 153/39 file after the snapshot 2023-10-30T12:48:05.628 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/.snap/snap-subdir-test 2023-10-30T12:48:05.648 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/30 file after the snapshot 2023-10-30T12:48:05.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/31 file after the snapshot 2023-10-30T12:48:05.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/32 file after the snapshot 2023-10-30T12:48:05.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/33 file after the snapshot 2023-10-30T12:48:05.651 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/34 file after the snapshot 2023-10-30T12:48:05.651 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/35 file after the snapshot 2023-10-30T12:48:05.652 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/36 file after the snapshot 2023-10-30T12:48:05.653 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/37 file after the snapshot 2023-10-30T12:48:05.654 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/38 file after the snapshot 2023-10-30T12:48:05.654 INFO:tasks.workunit.client.0.smithi008.stdout: create 154/39 file after the snapshot 2023-10-30T12:48:05.655 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/.snap/snap-subdir-test 2023-10-30T12:48:05.675 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/30 file after the snapshot 2023-10-30T12:48:05.676 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/31 file after the snapshot 2023-10-30T12:48:05.676 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/32 file after the snapshot 2023-10-30T12:48:05.677 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/33 file after the snapshot 2023-10-30T12:48:05.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/34 file after the snapshot 2023-10-30T12:48:05.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/35 file after the snapshot 2023-10-30T12:48:05.679 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/36 file after the snapshot 2023-10-30T12:48:05.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/37 file after the snapshot 2023-10-30T12:48:05.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/38 file after the snapshot 2023-10-30T12:48:05.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 155/39 file after the snapshot 2023-10-30T12:48:05.682 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/.snap/snap-subdir-test 2023-10-30T12:48:05.701 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/30 file after the snapshot 2023-10-30T12:48:05.702 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/31 file after the snapshot 2023-10-30T12:48:05.703 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/32 file after the snapshot 2023-10-30T12:48:05.703 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/33 file after the snapshot 2023-10-30T12:48:05.704 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/34 file after the snapshot 2023-10-30T12:48:05.705 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/35 file after the snapshot 2023-10-30T12:48:05.706 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/36 file after the snapshot 2023-10-30T12:48:05.706 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/37 file after the snapshot 2023-10-30T12:48:05.707 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/38 file after the snapshot 2023-10-30T12:48:05.708 INFO:tasks.workunit.client.0.smithi008.stdout: create 156/39 file after the snapshot 2023-10-30T12:48:05.708 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/.snap/snap-subdir-test 2023-10-30T12:48:05.726 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/30 file after the snapshot 2023-10-30T12:48:05.727 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/31 file after the snapshot 2023-10-30T12:48:05.727 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/32 file after the snapshot 2023-10-30T12:48:05.728 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/33 file after the snapshot 2023-10-30T12:48:05.729 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/34 file after the snapshot 2023-10-30T12:48:05.729 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/35 file after the snapshot 2023-10-30T12:48:05.730 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/36 file after the snapshot 2023-10-30T12:48:05.731 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/37 file after the snapshot 2023-10-30T12:48:05.731 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/38 file after the snapshot 2023-10-30T12:48:05.732 INFO:tasks.workunit.client.0.smithi008.stdout: create 157/39 file after the snapshot 2023-10-30T12:48:05.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/.snap/snap-subdir-test 2023-10-30T12:48:05.753 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/30 file after the snapshot 2023-10-30T12:48:05.754 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/31 file after the snapshot 2023-10-30T12:48:05.755 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/32 file after the snapshot 2023-10-30T12:48:05.756 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/33 file after the snapshot 2023-10-30T12:48:05.757 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/34 file after the snapshot 2023-10-30T12:48:05.758 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/35 file after the snapshot 2023-10-30T12:48:05.759 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/36 file after the snapshot 2023-10-30T12:48:05.760 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/37 file after the snapshot 2023-10-30T12:48:05.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/38 file after the snapshot 2023-10-30T12:48:05.762 INFO:tasks.workunit.client.0.smithi008.stdout: create 158/39 file after the snapshot 2023-10-30T12:48:05.763 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/.snap/snap-subdir-test 2023-10-30T12:48:05.784 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/30 file after the snapshot 2023-10-30T12:48:05.785 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/31 file after the snapshot 2023-10-30T12:48:05.786 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/32 file after the snapshot 2023-10-30T12:48:05.786 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/33 file after the snapshot 2023-10-30T12:48:05.787 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/34 file after the snapshot 2023-10-30T12:48:05.788 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/35 file after the snapshot 2023-10-30T12:48:05.789 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/36 file after the snapshot 2023-10-30T12:48:05.789 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/37 file after the snapshot 2023-10-30T12:48:05.790 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/38 file after the snapshot 2023-10-30T12:48:05.791 INFO:tasks.workunit.client.0.smithi008.stdout: create 159/39 file after the snapshot 2023-10-30T12:48:05.791 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/.snap/snap-subdir-test 2023-10-30T12:48:05.811 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/30 file after the snapshot 2023-10-30T12:48:05.812 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/31 file after the snapshot 2023-10-30T12:48:05.813 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/32 file after the snapshot 2023-10-30T12:48:05.814 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/33 file after the snapshot 2023-10-30T12:48:05.815 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/34 file after the snapshot 2023-10-30T12:48:05.816 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/35 file after the snapshot 2023-10-30T12:48:05.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/36 file after the snapshot 2023-10-30T12:48:05.819 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/37 file after the snapshot 2023-10-30T12:48:05.820 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/38 file after the snapshot 2023-10-30T12:48:05.821 INFO:tasks.workunit.client.0.smithi008.stdout: create 160/39 file after the snapshot 2023-10-30T12:48:05.822 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/.snap/snap-subdir-test 2023-10-30T12:48:05.838 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/30 file after the snapshot 2023-10-30T12:48:05.840 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/31 file after the snapshot 2023-10-30T12:48:05.841 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/32 file after the snapshot 2023-10-30T12:48:05.842 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/33 file after the snapshot 2023-10-30T12:48:05.843 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/34 file after the snapshot 2023-10-30T12:48:05.844 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/35 file after the snapshot 2023-10-30T12:48:05.845 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/36 file after the snapshot 2023-10-30T12:48:05.846 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/37 file after the snapshot 2023-10-30T12:48:05.847 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/38 file after the snapshot 2023-10-30T12:48:05.848 INFO:tasks.workunit.client.0.smithi008.stdout: create 161/39 file after the snapshot 2023-10-30T12:48:05.849 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/.snap/snap-subdir-test 2023-10-30T12:48:05.866 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/30 file after the snapshot 2023-10-30T12:48:05.867 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/31 file after the snapshot 2023-10-30T12:48:05.868 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/32 file after the snapshot 2023-10-30T12:48:05.869 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/33 file after the snapshot 2023-10-30T12:48:05.869 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/34 file after the snapshot 2023-10-30T12:48:05.870 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/35 file after the snapshot 2023-10-30T12:48:05.871 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/36 file after the snapshot 2023-10-30T12:48:05.871 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/37 file after the snapshot 2023-10-30T12:48:05.872 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/38 file after the snapshot 2023-10-30T12:48:05.873 INFO:tasks.workunit.client.0.smithi008.stdout: create 162/39 file after the snapshot 2023-10-30T12:48:05.874 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/.snap/snap-subdir-test 2023-10-30T12:48:05.892 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/30 file after the snapshot 2023-10-30T12:48:05.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/31 file after the snapshot 2023-10-30T12:48:05.894 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/32 file after the snapshot 2023-10-30T12:48:05.895 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/33 file after the snapshot 2023-10-30T12:48:05.896 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/34 file after the snapshot 2023-10-30T12:48:05.897 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/35 file after the snapshot 2023-10-30T12:48:05.898 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/36 file after the snapshot 2023-10-30T12:48:05.899 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/37 file after the snapshot 2023-10-30T12:48:05.900 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/38 file after the snapshot 2023-10-30T12:48:05.900 INFO:tasks.workunit.client.0.smithi008.stdout: create 163/39 file after the snapshot 2023-10-30T12:48:05.901 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/.snap/snap-subdir-test 2023-10-30T12:48:05.918 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/30 file after the snapshot 2023-10-30T12:48:05.919 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/31 file after the snapshot 2023-10-30T12:48:05.920 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/32 file after the snapshot 2023-10-30T12:48:05.921 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/33 file after the snapshot 2023-10-30T12:48:05.921 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/34 file after the snapshot 2023-10-30T12:48:05.922 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/35 file after the snapshot 2023-10-30T12:48:05.923 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/36 file after the snapshot 2023-10-30T12:48:05.923 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/37 file after the snapshot 2023-10-30T12:48:05.924 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/38 file after the snapshot 2023-10-30T12:48:05.925 INFO:tasks.workunit.client.0.smithi008.stdout: create 164/39 file after the snapshot 2023-10-30T12:48:05.925 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/.snap/snap-subdir-test 2023-10-30T12:48:05.947 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/30 file after the snapshot 2023-10-30T12:48:05.948 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/31 file after the snapshot 2023-10-30T12:48:05.949 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/32 file after the snapshot 2023-10-30T12:48:05.950 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/33 file after the snapshot 2023-10-30T12:48:05.951 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/34 file after the snapshot 2023-10-30T12:48:05.952 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/35 file after the snapshot 2023-10-30T12:48:05.953 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/36 file after the snapshot 2023-10-30T12:48:05.954 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/37 file after the snapshot 2023-10-30T12:48:05.955 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/38 file after the snapshot 2023-10-30T12:48:05.955 INFO:tasks.workunit.client.0.smithi008.stdout: create 165/39 file after the snapshot 2023-10-30T12:48:05.956 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/.snap/snap-subdir-test 2023-10-30T12:48:05.973 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/30 file after the snapshot 2023-10-30T12:48:05.974 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/31 file after the snapshot 2023-10-30T12:48:05.978 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/32 file after the snapshot 2023-10-30T12:48:05.978 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/33 file after the snapshot 2023-10-30T12:48:05.979 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/34 file after the snapshot 2023-10-30T12:48:05.979 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/35 file after the snapshot 2023-10-30T12:48:05.979 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/36 file after the snapshot 2023-10-30T12:48:05.979 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/37 file after the snapshot 2023-10-30T12:48:05.980 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/38 file after the snapshot 2023-10-30T12:48:05.980 INFO:tasks.workunit.client.0.smithi008.stdout: create 166/39 file after the snapshot 2023-10-30T12:48:05.981 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/.snap/snap-subdir-test 2023-10-30T12:48:06.000 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/30 file after the snapshot 2023-10-30T12:48:06.071 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/31 file after the snapshot 2023-10-30T12:48:06.268 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/32 file after the snapshot 2023-10-30T12:48:06.268 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/33 file after the snapshot 2023-10-30T12:48:06.269 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/34 file after the snapshot 2023-10-30T12:48:06.270 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/35 file after the snapshot 2023-10-30T12:48:06.271 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/36 file after the snapshot 2023-10-30T12:48:06.271 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/37 file after the snapshot 2023-10-30T12:48:06.272 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/38 file after the snapshot 2023-10-30T12:48:06.273 INFO:tasks.workunit.client.0.smithi008.stdout: create 167/39 file after the snapshot 2023-10-30T12:48:06.273 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/.snap/snap-subdir-test 2023-10-30T12:48:08.251 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/30 file after the snapshot 2023-10-30T12:48:08.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/31 file after the snapshot 2023-10-30T12:48:08.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/32 file after the snapshot 2023-10-30T12:48:08.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/33 file after the snapshot 2023-10-30T12:48:08.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/34 file after the snapshot 2023-10-30T12:48:08.253 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/35 file after the snapshot 2023-10-30T12:48:08.254 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/36 file after the snapshot 2023-10-30T12:48:08.254 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/37 file after the snapshot 2023-10-30T12:48:08.254 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/38 file after the snapshot 2023-10-30T12:48:08.255 INFO:tasks.workunit.client.0.smithi008.stdout: create 168/39 file after the snapshot 2023-10-30T12:48:08.255 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/.snap/snap-subdir-test 2023-10-30T12:48:08.274 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/30 file after the snapshot 2023-10-30T12:48:08.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/31 file after the snapshot 2023-10-30T12:48:08.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/32 file after the snapshot 2023-10-30T12:48:08.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/33 file after the snapshot 2023-10-30T12:48:08.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/34 file after the snapshot 2023-10-30T12:48:08.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/35 file after the snapshot 2023-10-30T12:48:08.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/36 file after the snapshot 2023-10-30T12:48:08.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/37 file after the snapshot 2023-10-30T12:48:08.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/38 file after the snapshot 2023-10-30T12:48:08.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 169/39 file after the snapshot 2023-10-30T12:48:08.278 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/.snap/snap-subdir-test 2023-10-30T12:48:08.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/30 file after the snapshot 2023-10-30T12:48:08.309 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/31 file after the snapshot 2023-10-30T12:48:08.309 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/32 file after the snapshot 2023-10-30T12:48:08.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/33 file after the snapshot 2023-10-30T12:48:08.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/34 file after the snapshot 2023-10-30T12:48:08.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/35 file after the snapshot 2023-10-30T12:48:08.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/36 file after the snapshot 2023-10-30T12:48:08.310 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/37 file after the snapshot 2023-10-30T12:48:08.311 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/38 file after the snapshot 2023-10-30T12:48:08.311 INFO:tasks.workunit.client.0.smithi008.stdout: create 170/39 file after the snapshot 2023-10-30T12:48:08.311 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/.snap/snap-subdir-test 2023-10-30T12:48:08.605 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/30 file after the snapshot 2023-10-30T12:48:08.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/31 file after the snapshot 2023-10-30T12:48:08.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/32 file after the snapshot 2023-10-30T12:48:08.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/33 file after the snapshot 2023-10-30T12:48:08.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/34 file after the snapshot 2023-10-30T12:48:08.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/35 file after the snapshot 2023-10-30T12:48:08.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/36 file after the snapshot 2023-10-30T12:48:08.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/37 file after the snapshot 2023-10-30T12:48:08.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/38 file after the snapshot 2023-10-30T12:48:08.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 171/39 file after the snapshot 2023-10-30T12:48:08.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/.snap/snap-subdir-test 2023-10-30T12:48:08.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/30 file after the snapshot 2023-10-30T12:48:08.637 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/31 file after the snapshot 2023-10-30T12:48:08.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/32 file after the snapshot 2023-10-30T12:48:08.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/33 file after the snapshot 2023-10-30T12:48:08.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/34 file after the snapshot 2023-10-30T12:48:08.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/35 file after the snapshot 2023-10-30T12:48:08.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/36 file after the snapshot 2023-10-30T12:48:08.639 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/37 file after the snapshot 2023-10-30T12:48:08.639 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/38 file after the snapshot 2023-10-30T12:48:08.639 INFO:tasks.workunit.client.0.smithi008.stdout: create 172/39 file after the snapshot 2023-10-30T12:48:08.639 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/.snap/snap-subdir-test 2023-10-30T12:48:09.161 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/30 file after the snapshot 2023-10-30T12:48:09.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/31 file after the snapshot 2023-10-30T12:48:09.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/32 file after the snapshot 2023-10-30T12:48:09.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/33 file after the snapshot 2023-10-30T12:48:09.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/34 file after the snapshot 2023-10-30T12:48:09.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/35 file after the snapshot 2023-10-30T12:48:09.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/36 file after the snapshot 2023-10-30T12:48:09.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/37 file after the snapshot 2023-10-30T12:48:09.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/38 file after the snapshot 2023-10-30T12:48:09.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 173/39 file after the snapshot 2023-10-30T12:48:09.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/.snap/snap-subdir-test 2023-10-30T12:48:09.631 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/30 file after the snapshot 2023-10-30T12:48:09.633 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/31 file after the snapshot 2023-10-30T12:48:09.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/32 file after the snapshot 2023-10-30T12:48:09.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/33 file after the snapshot 2023-10-30T12:48:09.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/34 file after the snapshot 2023-10-30T12:48:09.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/35 file after the snapshot 2023-10-30T12:48:09.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/36 file after the snapshot 2023-10-30T12:48:09.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/37 file after the snapshot 2023-10-30T12:48:09.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/38 file after the snapshot 2023-10-30T12:48:09.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 174/39 file after the snapshot 2023-10-30T12:48:09.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/.snap/snap-subdir-test 2023-10-30T12:48:09.665 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/30 file after the snapshot 2023-10-30T12:48:09.667 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/31 file after the snapshot 2023-10-30T12:48:09.667 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/32 file after the snapshot 2023-10-30T12:48:09.667 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/33 file after the snapshot 2023-10-30T12:48:09.668 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/34 file after the snapshot 2023-10-30T12:48:09.668 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/35 file after the snapshot 2023-10-30T12:48:09.668 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/36 file after the snapshot 2023-10-30T12:48:09.668 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/37 file after the snapshot 2023-10-30T12:48:09.668 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/38 file after the snapshot 2023-10-30T12:48:09.669 INFO:tasks.workunit.client.0.smithi008.stdout: create 175/39 file after the snapshot 2023-10-30T12:48:09.669 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/.snap/snap-subdir-test 2023-10-30T12:48:09.692 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/30 file after the snapshot 2023-10-30T12:48:09.694 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/31 file after the snapshot 2023-10-30T12:48:09.694 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/32 file after the snapshot 2023-10-30T12:48:09.694 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/33 file after the snapshot 2023-10-30T12:48:09.694 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/34 file after the snapshot 2023-10-30T12:48:09.695 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/35 file after the snapshot 2023-10-30T12:48:09.695 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/36 file after the snapshot 2023-10-30T12:48:09.695 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/37 file after the snapshot 2023-10-30T12:48:09.695 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/38 file after the snapshot 2023-10-30T12:48:09.695 INFO:tasks.workunit.client.0.smithi008.stdout: create 176/39 file after the snapshot 2023-10-30T12:48:09.696 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/.snap/snap-subdir-test 2023-10-30T12:48:09.716 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/30 file after the snapshot 2023-10-30T12:48:09.717 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/31 file after the snapshot 2023-10-30T12:48:09.718 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/32 file after the snapshot 2023-10-30T12:48:09.718 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/33 file after the snapshot 2023-10-30T12:48:09.718 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/34 file after the snapshot 2023-10-30T12:48:09.718 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/35 file after the snapshot 2023-10-30T12:48:09.718 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/36 file after the snapshot 2023-10-30T12:48:09.719 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/37 file after the snapshot 2023-10-30T12:48:09.719 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/38 file after the snapshot 2023-10-30T12:48:09.719 INFO:tasks.workunit.client.0.smithi008.stdout: create 177/39 file after the snapshot 2023-10-30T12:48:09.719 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/.snap/snap-subdir-test 2023-10-30T12:48:09.745 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/30 file after the snapshot 2023-10-30T12:48:09.746 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/31 file after the snapshot 2023-10-30T12:48:09.747 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/32 file after the snapshot 2023-10-30T12:48:09.747 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/33 file after the snapshot 2023-10-30T12:48:09.747 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/34 file after the snapshot 2023-10-30T12:48:09.747 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/35 file after the snapshot 2023-10-30T12:48:09.748 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/36 file after the snapshot 2023-10-30T12:48:09.748 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/37 file after the snapshot 2023-10-30T12:48:09.748 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/38 file after the snapshot 2023-10-30T12:48:09.748 INFO:tasks.workunit.client.0.smithi008.stdout: create 178/39 file after the snapshot 2023-10-30T12:48:09.748 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/.snap/snap-subdir-test 2023-10-30T12:48:10.698 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/30 file after the snapshot 2023-10-30T12:48:10.701 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/31 file after the snapshot 2023-10-30T12:48:10.701 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/32 file after the snapshot 2023-10-30T12:48:10.701 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/33 file after the snapshot 2023-10-30T12:48:10.702 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/34 file after the snapshot 2023-10-30T12:48:10.702 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/35 file after the snapshot 2023-10-30T12:48:10.703 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/36 file after the snapshot 2023-10-30T12:48:10.703 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/37 file after the snapshot 2023-10-30T12:48:10.703 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/38 file after the snapshot 2023-10-30T12:48:10.704 INFO:tasks.workunit.client.0.smithi008.stdout: create 179/39 file after the snapshot 2023-10-30T12:48:10.704 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/.snap/snap-subdir-test 2023-10-30T12:48:10.729 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/30 file after the snapshot 2023-10-30T12:48:10.732 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/31 file after the snapshot 2023-10-30T12:48:10.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/32 file after the snapshot 2023-10-30T12:48:10.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/33 file after the snapshot 2023-10-30T12:48:10.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/34 file after the snapshot 2023-10-30T12:48:10.734 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/35 file after the snapshot 2023-10-30T12:48:10.734 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/36 file after the snapshot 2023-10-30T12:48:10.735 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/37 file after the snapshot 2023-10-30T12:48:10.735 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/38 file after the snapshot 2023-10-30T12:48:10.735 INFO:tasks.workunit.client.0.smithi008.stdout: create 180/39 file after the snapshot 2023-10-30T12:48:10.736 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/.snap/snap-subdir-test 2023-10-30T12:48:10.752 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/30 file after the snapshot 2023-10-30T12:48:10.754 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/31 file after the snapshot 2023-10-30T12:48:10.755 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/32 file after the snapshot 2023-10-30T12:48:10.755 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/33 file after the snapshot 2023-10-30T12:48:10.756 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/34 file after the snapshot 2023-10-30T12:48:10.756 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/35 file after the snapshot 2023-10-30T12:48:10.757 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/36 file after the snapshot 2023-10-30T12:48:10.757 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/37 file after the snapshot 2023-10-30T12:48:10.758 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/38 file after the snapshot 2023-10-30T12:48:10.758 INFO:tasks.workunit.client.0.smithi008.stdout: create 181/39 file after the snapshot 2023-10-30T12:48:10.758 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/.snap/snap-subdir-test 2023-10-30T12:48:10.776 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/30 file after the snapshot 2023-10-30T12:48:10.777 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/31 file after the snapshot 2023-10-30T12:48:10.777 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/32 file after the snapshot 2023-10-30T12:48:10.778 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/33 file after the snapshot 2023-10-30T12:48:10.778 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/34 file after the snapshot 2023-10-30T12:48:10.778 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/35 file after the snapshot 2023-10-30T12:48:10.778 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/36 file after the snapshot 2023-10-30T12:48:10.778 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/37 file after the snapshot 2023-10-30T12:48:10.779 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/38 file after the snapshot 2023-10-30T12:48:10.779 INFO:tasks.workunit.client.0.smithi008.stdout: create 182/39 file after the snapshot 2023-10-30T12:48:10.779 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/.snap/snap-subdir-test 2023-10-30T12:48:10.799 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/30 file after the snapshot 2023-10-30T12:48:10.801 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/31 file after the snapshot 2023-10-30T12:48:10.802 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/32 file after the snapshot 2023-10-30T12:48:10.802 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/33 file after the snapshot 2023-10-30T12:48:10.803 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/34 file after the snapshot 2023-10-30T12:48:10.803 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/35 file after the snapshot 2023-10-30T12:48:10.804 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/36 file after the snapshot 2023-10-30T12:48:10.804 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/37 file after the snapshot 2023-10-30T12:48:10.804 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/38 file after the snapshot 2023-10-30T12:48:10.805 INFO:tasks.workunit.client.0.smithi008.stdout: create 183/39 file after the snapshot 2023-10-30T12:48:10.805 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/.snap/snap-subdir-test 2023-10-30T12:48:10.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/30 file after the snapshot 2023-10-30T12:48:10.831 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/31 file after the snapshot 2023-10-30T12:48:10.831 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/32 file after the snapshot 2023-10-30T12:48:10.831 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/33 file after the snapshot 2023-10-30T12:48:10.832 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/34 file after the snapshot 2023-10-30T12:48:10.832 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/35 file after the snapshot 2023-10-30T12:48:10.832 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/36 file after the snapshot 2023-10-30T12:48:10.833 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/37 file after the snapshot 2023-10-30T12:48:10.833 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/38 file after the snapshot 2023-10-30T12:48:10.833 INFO:tasks.workunit.client.0.smithi008.stdout: create 184/39 file after the snapshot 2023-10-30T12:48:10.833 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/.snap/snap-subdir-test 2023-10-30T12:48:10.851 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/30 file after the snapshot 2023-10-30T12:48:10.852 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/31 file after the snapshot 2023-10-30T12:48:10.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/32 file after the snapshot 2023-10-30T12:48:10.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/33 file after the snapshot 2023-10-30T12:48:10.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/34 file after the snapshot 2023-10-30T12:48:10.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/35 file after the snapshot 2023-10-30T12:48:10.854 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/36 file after the snapshot 2023-10-30T12:48:10.854 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/37 file after the snapshot 2023-10-30T12:48:10.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/38 file after the snapshot 2023-10-30T12:48:10.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 185/39 file after the snapshot 2023-10-30T12:48:10.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/.snap/snap-subdir-test 2023-10-30T12:48:10.880 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/30 file after the snapshot 2023-10-30T12:48:10.882 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/31 file after the snapshot 2023-10-30T12:48:10.882 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/32 file after the snapshot 2023-10-30T12:48:10.882 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/33 file after the snapshot 2023-10-30T12:48:10.883 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/34 file after the snapshot 2023-10-30T12:48:10.883 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/35 file after the snapshot 2023-10-30T12:48:10.883 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/36 file after the snapshot 2023-10-30T12:48:10.883 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/37 file after the snapshot 2023-10-30T12:48:10.884 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/38 file after the snapshot 2023-10-30T12:48:10.884 INFO:tasks.workunit.client.0.smithi008.stdout: create 186/39 file after the snapshot 2023-10-30T12:48:10.885 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/.snap/snap-subdir-test 2023-10-30T12:48:11.159 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/30 file after the snapshot 2023-10-30T12:48:11.161 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/31 file after the snapshot 2023-10-30T12:48:11.161 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/32 file after the snapshot 2023-10-30T12:48:11.161 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/33 file after the snapshot 2023-10-30T12:48:11.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/34 file after the snapshot 2023-10-30T12:48:11.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/35 file after the snapshot 2023-10-30T12:48:11.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/36 file after the snapshot 2023-10-30T12:48:11.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/37 file after the snapshot 2023-10-30T12:48:11.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/38 file after the snapshot 2023-10-30T12:48:11.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 187/39 file after the snapshot 2023-10-30T12:48:11.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/.snap/snap-subdir-test 2023-10-30T12:48:11.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/30 file after the snapshot 2023-10-30T12:48:11.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/31 file after the snapshot 2023-10-30T12:48:11.189 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/32 file after the snapshot 2023-10-30T12:48:11.189 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/33 file after the snapshot 2023-10-30T12:48:11.190 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/34 file after the snapshot 2023-10-30T12:48:11.191 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/35 file after the snapshot 2023-10-30T12:48:11.191 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/36 file after the snapshot 2023-10-30T12:48:11.192 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/37 file after the snapshot 2023-10-30T12:48:11.193 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/38 file after the snapshot 2023-10-30T12:48:11.194 INFO:tasks.workunit.client.0.smithi008.stdout: create 188/39 file after the snapshot 2023-10-30T12:48:11.194 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/.snap/snap-subdir-test 2023-10-30T12:48:11.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/30 file after the snapshot 2023-10-30T12:48:11.213 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/31 file after the snapshot 2023-10-30T12:48:11.214 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/32 file after the snapshot 2023-10-30T12:48:11.214 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/33 file after the snapshot 2023-10-30T12:48:11.215 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/34 file after the snapshot 2023-10-30T12:48:11.216 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/35 file after the snapshot 2023-10-30T12:48:11.216 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/36 file after the snapshot 2023-10-30T12:48:11.217 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/37 file after the snapshot 2023-10-30T12:48:11.218 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/38 file after the snapshot 2023-10-30T12:48:11.218 INFO:tasks.workunit.client.0.smithi008.stdout: create 189/39 file after the snapshot 2023-10-30T12:48:11.219 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/.snap/snap-subdir-test 2023-10-30T12:48:11.238 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/30 file after the snapshot 2023-10-30T12:48:11.243 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/31 file after the snapshot 2023-10-30T12:48:11.245 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/32 file after the snapshot 2023-10-30T12:48:11.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/33 file after the snapshot 2023-10-30T12:48:11.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/34 file after the snapshot 2023-10-30T12:48:11.251 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/35 file after the snapshot 2023-10-30T12:48:11.254 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/36 file after the snapshot 2023-10-30T12:48:11.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/37 file after the snapshot 2023-10-30T12:48:11.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/38 file after the snapshot 2023-10-30T12:48:11.261 INFO:tasks.workunit.client.0.smithi008.stdout: create 190/39 file after the snapshot 2023-10-30T12:48:11.262 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/.snap/snap-subdir-test 2023-10-30T12:48:11.287 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/30 file after the snapshot 2023-10-30T12:48:11.289 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/31 file after the snapshot 2023-10-30T12:48:11.291 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/32 file after the snapshot 2023-10-30T12:48:11.292 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/33 file after the snapshot 2023-10-30T12:48:11.294 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/34 file after the snapshot 2023-10-30T12:48:11.296 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/35 file after the snapshot 2023-10-30T12:48:11.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/36 file after the snapshot 2023-10-30T12:48:11.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/37 file after the snapshot 2023-10-30T12:48:11.301 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/38 file after the snapshot 2023-10-30T12:48:11.305 INFO:tasks.workunit.client.0.smithi008.stdout: create 191/39 file after the snapshot 2023-10-30T12:48:11.307 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/.snap/snap-subdir-test 2023-10-30T12:48:11.324 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/30 file after the snapshot 2023-10-30T12:48:11.326 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/31 file after the snapshot 2023-10-30T12:48:11.328 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/32 file after the snapshot 2023-10-30T12:48:11.329 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/33 file after the snapshot 2023-10-30T12:48:11.331 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/34 file after the snapshot 2023-10-30T12:48:11.333 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/35 file after the snapshot 2023-10-30T12:48:11.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/36 file after the snapshot 2023-10-30T12:48:11.336 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/37 file after the snapshot 2023-10-30T12:48:11.338 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/38 file after the snapshot 2023-10-30T12:48:11.340 INFO:tasks.workunit.client.0.smithi008.stdout: create 192/39 file after the snapshot 2023-10-30T12:48:11.342 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/.snap/snap-subdir-test 2023-10-30T12:48:11.360 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/30 file after the snapshot 2023-10-30T12:48:11.362 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/31 file after the snapshot 2023-10-30T12:48:11.364 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/32 file after the snapshot 2023-10-30T12:48:11.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/33 file after the snapshot 2023-10-30T12:48:11.367 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/34 file after the snapshot 2023-10-30T12:48:11.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/35 file after the snapshot 2023-10-30T12:48:11.371 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/36 file after the snapshot 2023-10-30T12:48:11.373 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/37 file after the snapshot 2023-10-30T12:48:11.374 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/38 file after the snapshot 2023-10-30T12:48:11.376 INFO:tasks.workunit.client.0.smithi008.stdout: create 193/39 file after the snapshot 2023-10-30T12:48:11.378 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/.snap/snap-subdir-test 2023-10-30T12:48:12.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/30 file after the snapshot 2023-10-30T12:48:12.140 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/31 file after the snapshot 2023-10-30T12:48:12.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/32 file after the snapshot 2023-10-30T12:48:12.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/33 file after the snapshot 2023-10-30T12:48:12.146 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/34 file after the snapshot 2023-10-30T12:48:12.148 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/35 file after the snapshot 2023-10-30T12:48:12.150 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/36 file after the snapshot 2023-10-30T12:48:12.153 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/37 file after the snapshot 2023-10-30T12:48:12.154 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/38 file after the snapshot 2023-10-30T12:48:12.156 INFO:tasks.workunit.client.0.smithi008.stdout: create 194/39 file after the snapshot 2023-10-30T12:48:12.158 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/.snap/snap-subdir-test 2023-10-30T12:48:12.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/30 file after the snapshot 2023-10-30T12:48:12.279 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/31 file after the snapshot 2023-10-30T12:48:12.280 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/32 file after the snapshot 2023-10-30T12:48:12.282 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/33 file after the snapshot 2023-10-30T12:48:12.536 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/34 file after the snapshot 2023-10-30T12:48:12.538 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/35 file after the snapshot 2023-10-30T12:48:12.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/36 file after the snapshot 2023-10-30T12:48:12.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/37 file after the snapshot 2023-10-30T12:48:12.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/38 file after the snapshot 2023-10-30T12:48:12.546 INFO:tasks.workunit.client.0.smithi008.stdout: create 195/39 file after the snapshot 2023-10-30T12:48:12.801 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/.snap/snap-subdir-test 2023-10-30T12:48:12.827 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/30 file after the snapshot 2023-10-30T12:48:12.830 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/31 file after the snapshot 2023-10-30T12:48:12.832 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/32 file after the snapshot 2023-10-30T12:48:12.835 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/33 file after the snapshot 2023-10-30T12:48:12.838 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/34 file after the snapshot 2023-10-30T12:48:12.840 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/35 file after the snapshot 2023-10-30T12:48:12.843 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/36 file after the snapshot 2023-10-30T12:48:12.846 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/37 file after the snapshot 2023-10-30T12:48:12.848 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/38 file after the snapshot 2023-10-30T12:48:12.852 INFO:tasks.workunit.client.0.smithi008.stdout: create 196/39 file after the snapshot 2023-10-30T12:48:12.854 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/.snap/snap-subdir-test 2023-10-30T12:48:12.880 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/30 file after the snapshot 2023-10-30T12:48:12.882 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/31 file after the snapshot 2023-10-30T12:48:12.885 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/32 file after the snapshot 2023-10-30T12:48:12.887 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/33 file after the snapshot 2023-10-30T12:48:12.890 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/34 file after the snapshot 2023-10-30T12:48:12.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/35 file after the snapshot 2023-10-30T12:48:12.895 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/36 file after the snapshot 2023-10-30T12:48:12.898 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/37 file after the snapshot 2023-10-30T12:48:12.901 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/38 file after the snapshot 2023-10-30T12:48:12.903 INFO:tasks.workunit.client.0.smithi008.stdout: create 197/39 file after the snapshot 2023-10-30T12:48:12.906 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/.snap/snap-subdir-test 2023-10-30T12:48:12.930 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/30 file after the snapshot 2023-10-30T12:48:12.933 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/31 file after the snapshot 2023-10-30T12:48:12.936 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/32 file after the snapshot 2023-10-30T12:48:12.938 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/33 file after the snapshot 2023-10-30T12:48:12.942 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/34 file after the snapshot 2023-10-30T12:48:12.944 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/35 file after the snapshot 2023-10-30T12:48:12.947 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/36 file after the snapshot 2023-10-30T12:48:12.949 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/37 file after the snapshot 2023-10-30T12:48:12.952 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/38 file after the snapshot 2023-10-30T12:48:12.954 INFO:tasks.workunit.client.0.smithi008.stdout: create 198/39 file after the snapshot 2023-10-30T12:48:12.957 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/.snap/snap-subdir-test 2023-10-30T12:48:12.974 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/30 file after the snapshot 2023-10-30T12:48:12.977 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/31 file after the snapshot 2023-10-30T12:48:12.980 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/32 file after the snapshot 2023-10-30T12:48:12.982 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/33 file after the snapshot 2023-10-30T12:48:12.985 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/34 file after the snapshot 2023-10-30T12:48:12.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/35 file after the snapshot 2023-10-30T12:48:12.991 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/36 file after the snapshot 2023-10-30T12:48:12.994 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/37 file after the snapshot 2023-10-30T12:48:12.997 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/38 file after the snapshot 2023-10-30T12:48:12.999 INFO:tasks.workunit.client.0.smithi008.stdout: create 199/39 file after the snapshot 2023-10-30T12:48:13.002 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/.snap/snap-subdir-test 2023-10-30T12:48:13.020 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/30 file after the snapshot 2023-10-30T12:48:13.022 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/31 file after the snapshot 2023-10-30T12:48:13.022 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/32 file after the snapshot 2023-10-30T12:48:13.023 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/33 file after the snapshot 2023-10-30T12:48:13.023 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/34 file after the snapshot 2023-10-30T12:48:13.023 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/35 file after the snapshot 2023-10-30T12:48:13.023 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/36 file after the snapshot 2023-10-30T12:48:13.023 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/37 file after the snapshot 2023-10-30T12:48:13.024 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/38 file after the snapshot 2023-10-30T12:48:13.024 INFO:tasks.workunit.client.0.smithi008.stdout: create 200/39 file after the snapshot 2023-10-30T12:48:13.024 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/.snap/snap-subdir-test 2023-10-30T12:48:13.043 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/30 file after the snapshot 2023-10-30T12:48:13.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/31 file after the snapshot 2023-10-30T12:48:13.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/32 file after the snapshot 2023-10-30T12:48:13.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/33 file after the snapshot 2023-10-30T12:48:13.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/34 file after the snapshot 2023-10-30T12:48:13.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/35 file after the snapshot 2023-10-30T12:48:13.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/36 file after the snapshot 2023-10-30T12:48:13.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/37 file after the snapshot 2023-10-30T12:48:13.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/38 file after the snapshot 2023-10-30T12:48:13.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 201/39 file after the snapshot 2023-10-30T12:48:13.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/.snap/snap-subdir-test 2023-10-30T12:48:13.067 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/30 file after the snapshot 2023-10-30T12:48:13.069 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/31 file after the snapshot 2023-10-30T12:48:13.069 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/32 file after the snapshot 2023-10-30T12:48:13.069 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/33 file after the snapshot 2023-10-30T12:48:13.069 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/34 file after the snapshot 2023-10-30T12:48:13.070 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/35 file after the snapshot 2023-10-30T12:48:13.070 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/36 file after the snapshot 2023-10-30T12:48:13.070 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/37 file after the snapshot 2023-10-30T12:48:13.070 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/38 file after the snapshot 2023-10-30T12:48:13.070 INFO:tasks.workunit.client.0.smithi008.stdout: create 202/39 file after the snapshot 2023-10-30T12:48:13.071 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/.snap/snap-subdir-test 2023-10-30T12:48:13.346 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/30 file after the snapshot 2023-10-30T12:48:13.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/31 file after the snapshot 2023-10-30T12:48:13.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/32 file after the snapshot 2023-10-30T12:48:13.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/33 file after the snapshot 2023-10-30T12:48:13.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/34 file after the snapshot 2023-10-30T12:48:13.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/35 file after the snapshot 2023-10-30T12:48:13.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/36 file after the snapshot 2023-10-30T12:48:13.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/37 file after the snapshot 2023-10-30T12:48:13.349 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/38 file after the snapshot 2023-10-30T12:48:13.349 INFO:tasks.workunit.client.0.smithi008.stdout: create 203/39 file after the snapshot 2023-10-30T12:48:13.349 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/.snap/snap-subdir-test 2023-10-30T12:48:13.366 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/30 file after the snapshot 2023-10-30T12:48:13.368 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/31 file after the snapshot 2023-10-30T12:48:13.368 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/32 file after the snapshot 2023-10-30T12:48:13.368 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/33 file after the snapshot 2023-10-30T12:48:13.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/34 file after the snapshot 2023-10-30T12:48:13.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/35 file after the snapshot 2023-10-30T12:48:13.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/36 file after the snapshot 2023-10-30T12:48:13.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/37 file after the snapshot 2023-10-30T12:48:13.369 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/38 file after the snapshot 2023-10-30T12:48:13.370 INFO:tasks.workunit.client.0.smithi008.stdout: create 204/39 file after the snapshot 2023-10-30T12:48:13.370 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/.snap/snap-subdir-test 2023-10-30T12:48:14.106 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/30 file after the snapshot 2023-10-30T12:48:14.108 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/31 file after the snapshot 2023-10-30T12:48:14.108 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/32 file after the snapshot 2023-10-30T12:48:14.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/33 file after the snapshot 2023-10-30T12:48:14.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/34 file after the snapshot 2023-10-30T12:48:14.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/35 file after the snapshot 2023-10-30T12:48:14.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/36 file after the snapshot 2023-10-30T12:48:14.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/37 file after the snapshot 2023-10-30T12:48:14.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/38 file after the snapshot 2023-10-30T12:48:14.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 205/39 file after the snapshot 2023-10-30T12:48:14.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/.snap/snap-subdir-test 2023-10-30T12:48:14.131 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/30 file after the snapshot 2023-10-30T12:48:14.132 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/31 file after the snapshot 2023-10-30T12:48:14.133 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/32 file after the snapshot 2023-10-30T12:48:14.133 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/33 file after the snapshot 2023-10-30T12:48:14.133 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/34 file after the snapshot 2023-10-30T12:48:14.133 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/35 file after the snapshot 2023-10-30T12:48:14.134 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/36 file after the snapshot 2023-10-30T12:48:14.134 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/37 file after the snapshot 2023-10-30T12:48:14.134 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/38 file after the snapshot 2023-10-30T12:48:14.134 INFO:tasks.workunit.client.0.smithi008.stdout: create 206/39 file after the snapshot 2023-10-30T12:48:14.134 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/.snap/snap-subdir-test 2023-10-30T12:48:14.244 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:14.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:15.067 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/30 file after the snapshot 2023-10-30T12:48:15.072 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/31 file after the snapshot 2023-10-30T12:48:15.073 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/32 file after the snapshot 2023-10-30T12:48:15.073 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/33 file after the snapshot 2023-10-30T12:48:15.073 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/34 file after the snapshot 2023-10-30T12:48:15.073 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/35 file after the snapshot 2023-10-30T12:48:15.074 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/36 file after the snapshot 2023-10-30T12:48:15.074 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/37 file after the snapshot 2023-10-30T12:48:15.074 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/38 file after the snapshot 2023-10-30T12:48:15.075 INFO:tasks.workunit.client.0.smithi008.stdout: create 207/39 file after the snapshot 2023-10-30T12:48:15.075 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/.snap/snap-subdir-test 2023-10-30T12:48:15.106 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/30 file after the snapshot 2023-10-30T12:48:15.108 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/31 file after the snapshot 2023-10-30T12:48:15.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/32 file after the snapshot 2023-10-30T12:48:15.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/33 file after the snapshot 2023-10-30T12:48:15.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/34 file after the snapshot 2023-10-30T12:48:15.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/35 file after the snapshot 2023-10-30T12:48:15.109 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/36 file after the snapshot 2023-10-30T12:48:15.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/37 file after the snapshot 2023-10-30T12:48:15.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/38 file after the snapshot 2023-10-30T12:48:15.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 208/39 file after the snapshot 2023-10-30T12:48:15.110 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/.snap/snap-subdir-test 2023-10-30T12:48:15.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/30 file after the snapshot 2023-10-30T12:48:15.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/31 file after the snapshot 2023-10-30T12:48:15.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/32 file after the snapshot 2023-10-30T12:48:15.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/33 file after the snapshot 2023-10-30T12:48:15.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/34 file after the snapshot 2023-10-30T12:48:15.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/35 file after the snapshot 2023-10-30T12:48:15.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/36 file after the snapshot 2023-10-30T12:48:15.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/37 file after the snapshot 2023-10-30T12:48:15.145 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/38 file after the snapshot 2023-10-30T12:48:15.145 INFO:tasks.workunit.client.0.smithi008.stdout: create 209/39 file after the snapshot 2023-10-30T12:48:15.145 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/.snap/snap-subdir-test 2023-10-30T12:48:15.447 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/30 file after the snapshot 2023-10-30T12:48:15.449 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/31 file after the snapshot 2023-10-30T12:48:15.449 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/32 file after the snapshot 2023-10-30T12:48:15.449 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/33 file after the snapshot 2023-10-30T12:48:15.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/34 file after the snapshot 2023-10-30T12:48:15.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/35 file after the snapshot 2023-10-30T12:48:15.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/36 file after the snapshot 2023-10-30T12:48:15.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/37 file after the snapshot 2023-10-30T12:48:15.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/38 file after the snapshot 2023-10-30T12:48:15.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 210/39 file after the snapshot 2023-10-30T12:48:15.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/.snap/snap-subdir-test 2023-10-30T12:48:15.469 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/30 file after the snapshot 2023-10-30T12:48:15.471 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/31 file after the snapshot 2023-10-30T12:48:15.471 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/32 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/33 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/34 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/35 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/36 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/37 file after the snapshot 2023-10-30T12:48:15.472 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/38 file after the snapshot 2023-10-30T12:48:15.473 INFO:tasks.workunit.client.0.smithi008.stdout: create 211/39 file after the snapshot 2023-10-30T12:48:15.473 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/.snap/snap-subdir-test 2023-10-30T12:48:15.497 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/30 file after the snapshot 2023-10-30T12:48:15.500 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/31 file after the snapshot 2023-10-30T12:48:15.500 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/32 file after the snapshot 2023-10-30T12:48:15.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/33 file after the snapshot 2023-10-30T12:48:15.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/34 file after the snapshot 2023-10-30T12:48:15.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/35 file after the snapshot 2023-10-30T12:48:15.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/36 file after the snapshot 2023-10-30T12:48:15.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/37 file after the snapshot 2023-10-30T12:48:15.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/38 file after the snapshot 2023-10-30T12:48:15.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 212/39 file after the snapshot 2023-10-30T12:48:15.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/.snap/snap-subdir-test 2023-10-30T12:48:15.522 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/30 file after the snapshot 2023-10-30T12:48:15.524 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/31 file after the snapshot 2023-10-30T12:48:15.524 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/32 file after the snapshot 2023-10-30T12:48:15.525 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/33 file after the snapshot 2023-10-30T12:48:15.525 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/34 file after the snapshot 2023-10-30T12:48:15.525 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/35 file after the snapshot 2023-10-30T12:48:15.525 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/36 file after the snapshot 2023-10-30T12:48:15.525 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/37 file after the snapshot 2023-10-30T12:48:15.526 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/38 file after the snapshot 2023-10-30T12:48:15.526 INFO:tasks.workunit.client.0.smithi008.stdout: create 213/39 file after the snapshot 2023-10-30T12:48:15.526 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/.snap/snap-subdir-test 2023-10-30T12:48:15.553 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/30 file after the snapshot 2023-10-30T12:48:15.554 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/31 file after the snapshot 2023-10-30T12:48:15.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/32 file after the snapshot 2023-10-30T12:48:15.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/33 file after the snapshot 2023-10-30T12:48:15.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/34 file after the snapshot 2023-10-30T12:48:15.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/35 file after the snapshot 2023-10-30T12:48:15.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/36 file after the snapshot 2023-10-30T12:48:15.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/37 file after the snapshot 2023-10-30T12:48:15.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/38 file after the snapshot 2023-10-30T12:48:15.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 214/39 file after the snapshot 2023-10-30T12:48:15.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/.snap/snap-subdir-test 2023-10-30T12:48:15.582 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/30 file after the snapshot 2023-10-30T12:48:15.583 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/31 file after the snapshot 2023-10-30T12:48:15.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/32 file after the snapshot 2023-10-30T12:48:15.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/33 file after the snapshot 2023-10-30T12:48:15.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/34 file after the snapshot 2023-10-30T12:48:15.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/35 file after the snapshot 2023-10-30T12:48:15.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/36 file after the snapshot 2023-10-30T12:48:15.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/37 file after the snapshot 2023-10-30T12:48:15.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/38 file after the snapshot 2023-10-30T12:48:15.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 215/39 file after the snapshot 2023-10-30T12:48:15.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/.snap/snap-subdir-test 2023-10-30T12:48:15.604 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/30 file after the snapshot 2023-10-30T12:48:15.605 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/31 file after the snapshot 2023-10-30T12:48:15.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/32 file after the snapshot 2023-10-30T12:48:15.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/33 file after the snapshot 2023-10-30T12:48:15.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/34 file after the snapshot 2023-10-30T12:48:15.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/35 file after the snapshot 2023-10-30T12:48:15.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/36 file after the snapshot 2023-10-30T12:48:15.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/37 file after the snapshot 2023-10-30T12:48:15.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/38 file after the snapshot 2023-10-30T12:48:15.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 216/39 file after the snapshot 2023-10-30T12:48:15.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/.snap/snap-subdir-test 2023-10-30T12:48:16.042 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/30 file after the snapshot 2023-10-30T12:48:16.044 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/31 file after the snapshot 2023-10-30T12:48:16.044 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/32 file after the snapshot 2023-10-30T12:48:16.044 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/33 file after the snapshot 2023-10-30T12:48:16.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/34 file after the snapshot 2023-10-30T12:48:16.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/35 file after the snapshot 2023-10-30T12:48:16.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/36 file after the snapshot 2023-10-30T12:48:16.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/37 file after the snapshot 2023-10-30T12:48:16.045 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/38 file after the snapshot 2023-10-30T12:48:16.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 217/39 file after the snapshot 2023-10-30T12:48:16.046 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/.snap/snap-subdir-test 2023-10-30T12:48:16.816 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/30 file after the snapshot 2023-10-30T12:48:16.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/31 file after the snapshot 2023-10-30T12:48:16.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/32 file after the snapshot 2023-10-30T12:48:16.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/33 file after the snapshot 2023-10-30T12:48:16.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/34 file after the snapshot 2023-10-30T12:48:16.818 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/35 file after the snapshot 2023-10-30T12:48:16.819 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/36 file after the snapshot 2023-10-30T12:48:16.819 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/37 file after the snapshot 2023-10-30T12:48:16.819 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/38 file after the snapshot 2023-10-30T12:48:16.819 INFO:tasks.workunit.client.0.smithi008.stdout: create 218/39 file after the snapshot 2023-10-30T12:48:16.820 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/.snap/snap-subdir-test 2023-10-30T12:48:16.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/30 file after the snapshot 2023-10-30T12:48:16.857 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/31 file after the snapshot 2023-10-30T12:48:16.857 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/32 file after the snapshot 2023-10-30T12:48:16.857 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/33 file after the snapshot 2023-10-30T12:48:16.858 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/34 file after the snapshot 2023-10-30T12:48:16.858 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/35 file after the snapshot 2023-10-30T12:48:16.858 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/36 file after the snapshot 2023-10-30T12:48:16.858 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/37 file after the snapshot 2023-10-30T12:48:16.858 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/38 file after the snapshot 2023-10-30T12:48:16.859 INFO:tasks.workunit.client.0.smithi008.stdout: create 219/39 file after the snapshot 2023-10-30T12:48:16.859 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/.snap/snap-subdir-test 2023-10-30T12:48:16.904 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/30 file after the snapshot 2023-10-30T12:48:16.907 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/31 file after the snapshot 2023-10-30T12:48:16.907 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/32 file after the snapshot 2023-10-30T12:48:16.908 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/33 file after the snapshot 2023-10-30T12:48:16.908 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/34 file after the snapshot 2023-10-30T12:48:16.908 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/35 file after the snapshot 2023-10-30T12:48:16.908 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/36 file after the snapshot 2023-10-30T12:48:16.908 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/37 file after the snapshot 2023-10-30T12:48:16.909 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/38 file after the snapshot 2023-10-30T12:48:16.909 INFO:tasks.workunit.client.0.smithi008.stdout: create 220/39 file after the snapshot 2023-10-30T12:48:16.909 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/.snap/snap-subdir-test 2023-10-30T12:48:17.096 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/30 file after the snapshot 2023-10-30T12:48:17.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/31 file after the snapshot 2023-10-30T12:48:17.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/32 file after the snapshot 2023-10-30T12:48:17.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/33 file after the snapshot 2023-10-30T12:48:17.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/34 file after the snapshot 2023-10-30T12:48:17.100 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/35 file after the snapshot 2023-10-30T12:48:17.100 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/36 file after the snapshot 2023-10-30T12:48:17.100 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/37 file after the snapshot 2023-10-30T12:48:17.100 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/38 file after the snapshot 2023-10-30T12:48:17.100 INFO:tasks.workunit.client.0.smithi008.stdout: create 221/39 file after the snapshot 2023-10-30T12:48:17.101 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/.snap/snap-subdir-test 2023-10-30T12:48:17.133 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/30 file after the snapshot 2023-10-30T12:48:17.136 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/31 file after the snapshot 2023-10-30T12:48:17.137 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/32 file after the snapshot 2023-10-30T12:48:17.137 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/33 file after the snapshot 2023-10-30T12:48:17.137 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/34 file after the snapshot 2023-10-30T12:48:17.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/35 file after the snapshot 2023-10-30T12:48:17.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/36 file after the snapshot 2023-10-30T12:48:17.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/37 file after the snapshot 2023-10-30T12:48:17.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/38 file after the snapshot 2023-10-30T12:48:17.139 INFO:tasks.workunit.client.0.smithi008.stdout: create 222/39 file after the snapshot 2023-10-30T12:48:17.139 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/.snap/snap-subdir-test 2023-10-30T12:48:17.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/30 file after the snapshot 2023-10-30T12:48:17.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/31 file after the snapshot 2023-10-30T12:48:17.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/32 file after the snapshot 2023-10-30T12:48:17.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/33 file after the snapshot 2023-10-30T12:48:17.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/34 file after the snapshot 2023-10-30T12:48:17.187 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/35 file after the snapshot 2023-10-30T12:48:17.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/36 file after the snapshot 2023-10-30T12:48:17.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/37 file after the snapshot 2023-10-30T12:48:17.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/38 file after the snapshot 2023-10-30T12:48:17.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 223/39 file after the snapshot 2023-10-30T12:48:17.188 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/.snap/snap-subdir-test 2023-10-30T12:48:17.219 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/30 file after the snapshot 2023-10-30T12:48:17.221 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/31 file after the snapshot 2023-10-30T12:48:17.222 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/32 file after the snapshot 2023-10-30T12:48:17.222 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/33 file after the snapshot 2023-10-30T12:48:17.222 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/34 file after the snapshot 2023-10-30T12:48:17.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/35 file after the snapshot 2023-10-30T12:48:17.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/36 file after the snapshot 2023-10-30T12:48:17.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/37 file after the snapshot 2023-10-30T12:48:17.224 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/38 file after the snapshot 2023-10-30T12:48:17.224 INFO:tasks.workunit.client.0.smithi008.stdout: create 224/39 file after the snapshot 2023-10-30T12:48:17.224 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/.snap/snap-subdir-test 2023-10-30T12:48:17.245 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/30 file after the snapshot 2023-10-30T12:48:17.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/31 file after the snapshot 2023-10-30T12:48:17.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/32 file after the snapshot 2023-10-30T12:48:17.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/33 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/34 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/35 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/36 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/37 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/38 file after the snapshot 2023-10-30T12:48:17.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 225/39 file after the snapshot 2023-10-30T12:48:17.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/.snap/snap-subdir-test 2023-10-30T12:48:17.271 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/30 file after the snapshot 2023-10-30T12:48:17.273 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/31 file after the snapshot 2023-10-30T12:48:17.274 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/32 file after the snapshot 2023-10-30T12:48:17.274 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/33 file after the snapshot 2023-10-30T12:48:17.275 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/34 file after the snapshot 2023-10-30T12:48:17.275 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/35 file after the snapshot 2023-10-30T12:48:17.275 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/36 file after the snapshot 2023-10-30T12:48:17.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/37 file after the snapshot 2023-10-30T12:48:17.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/38 file after the snapshot 2023-10-30T12:48:17.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 226/39 file after the snapshot 2023-10-30T12:48:17.277 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/.snap/snap-subdir-test 2023-10-30T12:48:17.296 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/30 file after the snapshot 2023-10-30T12:48:17.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/31 file after the snapshot 2023-10-30T12:48:17.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/32 file after the snapshot 2023-10-30T12:48:17.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/33 file after the snapshot 2023-10-30T12:48:17.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/34 file after the snapshot 2023-10-30T12:48:17.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/35 file after the snapshot 2023-10-30T12:48:17.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/36 file after the snapshot 2023-10-30T12:48:17.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/37 file after the snapshot 2023-10-30T12:48:17.300 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/38 file after the snapshot 2023-10-30T12:48:17.300 INFO:tasks.workunit.client.0.smithi008.stdout: create 227/39 file after the snapshot 2023-10-30T12:48:17.300 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/.snap/snap-subdir-test 2023-10-30T12:48:17.330 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/30 file after the snapshot 2023-10-30T12:48:17.333 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/31 file after the snapshot 2023-10-30T12:48:17.334 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/32 file after the snapshot 2023-10-30T12:48:17.334 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/33 file after the snapshot 2023-10-30T12:48:17.334 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/34 file after the snapshot 2023-10-30T12:48:17.334 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/35 file after the snapshot 2023-10-30T12:48:17.334 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/36 file after the snapshot 2023-10-30T12:48:17.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/37 file after the snapshot 2023-10-30T12:48:17.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/38 file after the snapshot 2023-10-30T12:48:17.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 228/39 file after the snapshot 2023-10-30T12:48:17.335 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/.snap/snap-subdir-test 2023-10-30T12:48:17.356 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/30 file after the snapshot 2023-10-30T12:48:17.358 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/31 file after the snapshot 2023-10-30T12:48:17.359 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/32 file after the snapshot 2023-10-30T12:48:17.359 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/33 file after the snapshot 2023-10-30T12:48:17.359 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/34 file after the snapshot 2023-10-30T12:48:17.359 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/35 file after the snapshot 2023-10-30T12:48:17.359 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/36 file after the snapshot 2023-10-30T12:48:17.360 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/37 file after the snapshot 2023-10-30T12:48:17.360 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/38 file after the snapshot 2023-10-30T12:48:17.360 INFO:tasks.workunit.client.0.smithi008.stdout: create 229/39 file after the snapshot 2023-10-30T12:48:17.360 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/.snap/snap-subdir-test 2023-10-30T12:48:17.388 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/30 file after the snapshot 2023-10-30T12:48:17.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/31 file after the snapshot 2023-10-30T12:48:17.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/32 file after the snapshot 2023-10-30T12:48:17.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/33 file after the snapshot 2023-10-30T12:48:17.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/34 file after the snapshot 2023-10-30T12:48:17.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/35 file after the snapshot 2023-10-30T12:48:17.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/36 file after the snapshot 2023-10-30T12:48:17.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/37 file after the snapshot 2023-10-30T12:48:17.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/38 file after the snapshot 2023-10-30T12:48:17.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 230/39 file after the snapshot 2023-10-30T12:48:17.393 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/.snap/snap-subdir-test 2023-10-30T12:48:17.411 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/30 file after the snapshot 2023-10-30T12:48:17.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/31 file after the snapshot 2023-10-30T12:48:17.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/32 file after the snapshot 2023-10-30T12:48:17.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/33 file after the snapshot 2023-10-30T12:48:17.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/34 file after the snapshot 2023-10-30T12:48:17.414 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/35 file after the snapshot 2023-10-30T12:48:17.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/36 file after the snapshot 2023-10-30T12:48:17.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/37 file after the snapshot 2023-10-30T12:48:17.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/38 file after the snapshot 2023-10-30T12:48:17.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 231/39 file after the snapshot 2023-10-30T12:48:17.415 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/.snap/snap-subdir-test 2023-10-30T12:48:17.447 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/30 file after the snapshot 2023-10-30T12:48:17.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/31 file after the snapshot 2023-10-30T12:48:17.450 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/32 file after the snapshot 2023-10-30T12:48:17.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/33 file after the snapshot 2023-10-30T12:48:17.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/34 file after the snapshot 2023-10-30T12:48:17.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/35 file after the snapshot 2023-10-30T12:48:17.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/36 file after the snapshot 2023-10-30T12:48:17.451 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/37 file after the snapshot 2023-10-30T12:48:17.452 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/38 file after the snapshot 2023-10-30T12:48:17.452 INFO:tasks.workunit.client.0.smithi008.stdout: create 232/39 file after the snapshot 2023-10-30T12:48:17.452 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/.snap/snap-subdir-test 2023-10-30T12:48:17.498 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/30 file after the snapshot 2023-10-30T12:48:17.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/31 file after the snapshot 2023-10-30T12:48:17.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/32 file after the snapshot 2023-10-30T12:48:17.501 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/33 file after the snapshot 2023-10-30T12:48:17.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/34 file after the snapshot 2023-10-30T12:48:17.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/35 file after the snapshot 2023-10-30T12:48:17.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/36 file after the snapshot 2023-10-30T12:48:17.502 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/37 file after the snapshot 2023-10-30T12:48:17.503 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/38 file after the snapshot 2023-10-30T12:48:17.503 INFO:tasks.workunit.client.0.smithi008.stdout: create 233/39 file after the snapshot 2023-10-30T12:48:17.503 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/.snap/snap-subdir-test 2023-10-30T12:48:18.148 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/30 file after the snapshot 2023-10-30T12:48:18.150 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/31 file after the snapshot 2023-10-30T12:48:18.150 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/32 file after the snapshot 2023-10-30T12:48:18.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/33 file after the snapshot 2023-10-30T12:48:18.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/34 file after the snapshot 2023-10-30T12:48:18.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/35 file after the snapshot 2023-10-30T12:48:18.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/36 file after the snapshot 2023-10-30T12:48:18.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/37 file after the snapshot 2023-10-30T12:48:18.152 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/38 file after the snapshot 2023-10-30T12:48:18.152 INFO:tasks.workunit.client.0.smithi008.stdout: create 234/39 file after the snapshot 2023-10-30T12:48:18.152 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/.snap/snap-subdir-test 2023-10-30T12:48:18.180 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/30 file after the snapshot 2023-10-30T12:48:18.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/31 file after the snapshot 2023-10-30T12:48:18.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/32 file after the snapshot 2023-10-30T12:48:18.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/33 file after the snapshot 2023-10-30T12:48:18.183 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/34 file after the snapshot 2023-10-30T12:48:18.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/35 file after the snapshot 2023-10-30T12:48:18.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/36 file after the snapshot 2023-10-30T12:48:18.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/37 file after the snapshot 2023-10-30T12:48:18.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/38 file after the snapshot 2023-10-30T12:48:18.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 235/39 file after the snapshot 2023-10-30T12:48:18.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/.snap/snap-subdir-test 2023-10-30T12:48:18.208 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/30 file after the snapshot 2023-10-30T12:48:18.210 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/31 file after the snapshot 2023-10-30T12:48:18.210 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/32 file after the snapshot 2023-10-30T12:48:18.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/33 file after the snapshot 2023-10-30T12:48:18.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/34 file after the snapshot 2023-10-30T12:48:18.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/35 file after the snapshot 2023-10-30T12:48:18.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/36 file after the snapshot 2023-10-30T12:48:18.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/37 file after the snapshot 2023-10-30T12:48:18.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/38 file after the snapshot 2023-10-30T12:48:18.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 236/39 file after the snapshot 2023-10-30T12:48:18.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/.snap/snap-subdir-test 2023-10-30T12:48:18.228 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/30 file after the snapshot 2023-10-30T12:48:18.230 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/31 file after the snapshot 2023-10-30T12:48:18.230 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/32 file after the snapshot 2023-10-30T12:48:18.230 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/33 file after the snapshot 2023-10-30T12:48:18.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/34 file after the snapshot 2023-10-30T12:48:18.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/35 file after the snapshot 2023-10-30T12:48:18.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/36 file after the snapshot 2023-10-30T12:48:18.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/37 file after the snapshot 2023-10-30T12:48:18.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/38 file after the snapshot 2023-10-30T12:48:18.232 INFO:tasks.workunit.client.0.smithi008.stdout: create 237/39 file after the snapshot 2023-10-30T12:48:18.232 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/.snap/snap-subdir-test 2023-10-30T12:48:18.256 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/30 file after the snapshot 2023-10-30T12:48:18.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/31 file after the snapshot 2023-10-30T12:48:18.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/32 file after the snapshot 2023-10-30T12:48:18.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/33 file after the snapshot 2023-10-30T12:48:18.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/34 file after the snapshot 2023-10-30T12:48:18.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/35 file after the snapshot 2023-10-30T12:48:18.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/36 file after the snapshot 2023-10-30T12:48:18.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/37 file after the snapshot 2023-10-30T12:48:18.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/38 file after the snapshot 2023-10-30T12:48:18.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 238/39 file after the snapshot 2023-10-30T12:48:18.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/.snap/snap-subdir-test 2023-10-30T12:48:18.468 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/30 file after the snapshot 2023-10-30T12:48:18.469 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/31 file after the snapshot 2023-10-30T12:48:18.469 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/32 file after the snapshot 2023-10-30T12:48:18.470 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/33 file after the snapshot 2023-10-30T12:48:18.470 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/34 file after the snapshot 2023-10-30T12:48:18.470 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/35 file after the snapshot 2023-10-30T12:48:18.470 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/36 file after the snapshot 2023-10-30T12:48:18.470 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/37 file after the snapshot 2023-10-30T12:48:18.471 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/38 file after the snapshot 2023-10-30T12:48:18.471 INFO:tasks.workunit.client.0.smithi008.stdout: create 239/39 file after the snapshot 2023-10-30T12:48:18.471 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/.snap/snap-subdir-test 2023-10-30T12:48:19.272 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/30 file after the snapshot 2023-10-30T12:48:19.399 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/31 file after the snapshot 2023-10-30T12:48:19.399 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/32 file after the snapshot 2023-10-30T12:48:19.399 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/33 file after the snapshot 2023-10-30T12:48:19.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/34 file after the snapshot 2023-10-30T12:48:19.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/35 file after the snapshot 2023-10-30T12:48:19.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/36 file after the snapshot 2023-10-30T12:48:19.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/37 file after the snapshot 2023-10-30T12:48:19.400 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/38 file after the snapshot 2023-10-30T12:48:19.401 INFO:tasks.workunit.client.0.smithi008.stdout: create 240/39 file after the snapshot 2023-10-30T12:48:19.401 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/.snap/snap-subdir-test 2023-10-30T12:48:20.353 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/30 file after the snapshot 2023-10-30T12:48:20.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/31 file after the snapshot 2023-10-30T12:48:20.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/32 file after the snapshot 2023-10-30T12:48:20.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/33 file after the snapshot 2023-10-30T12:48:20.608 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/34 file after the snapshot 2023-10-30T12:48:20.609 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/35 file after the snapshot 2023-10-30T12:48:20.609 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/36 file after the snapshot 2023-10-30T12:48:20.609 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/37 file after the snapshot 2023-10-30T12:48:20.609 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/38 file after the snapshot 2023-10-30T12:48:20.609 INFO:tasks.workunit.client.0.smithi008.stdout: create 241/39 file after the snapshot 2023-10-30T12:48:20.610 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/.snap/snap-subdir-test 2023-10-30T12:48:21.050 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/30 file after the snapshot 2023-10-30T12:48:21.052 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/31 file after the snapshot 2023-10-30T12:48:21.052 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/32 file after the snapshot 2023-10-30T12:48:21.052 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/33 file after the snapshot 2023-10-30T12:48:21.052 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/34 file after the snapshot 2023-10-30T12:48:21.052 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/35 file after the snapshot 2023-10-30T12:48:21.053 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/36 file after the snapshot 2023-10-30T12:48:21.053 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/37 file after the snapshot 2023-10-30T12:48:21.053 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/38 file after the snapshot 2023-10-30T12:48:21.053 INFO:tasks.workunit.client.0.smithi008.stdout: create 242/39 file after the snapshot 2023-10-30T12:48:21.053 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/.snap/snap-subdir-test 2023-10-30T12:48:21.529 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/30 file after the snapshot 2023-10-30T12:48:21.531 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/31 file after the snapshot 2023-10-30T12:48:21.531 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/32 file after the snapshot 2023-10-30T12:48:21.531 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/33 file after the snapshot 2023-10-30T12:48:21.532 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/34 file after the snapshot 2023-10-30T12:48:21.532 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/35 file after the snapshot 2023-10-30T12:48:21.532 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/36 file after the snapshot 2023-10-30T12:48:21.533 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/37 file after the snapshot 2023-10-30T12:48:21.533 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/38 file after the snapshot 2023-10-30T12:48:21.533 INFO:tasks.workunit.client.0.smithi008.stdout: create 243/39 file after the snapshot 2023-10-30T12:48:21.534 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/.snap/snap-subdir-test 2023-10-30T12:48:21.563 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/30 file after the snapshot 2023-10-30T12:48:21.564 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/31 file after the snapshot 2023-10-30T12:48:21.564 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/32 file after the snapshot 2023-10-30T12:48:21.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/33 file after the snapshot 2023-10-30T12:48:21.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/34 file after the snapshot 2023-10-30T12:48:21.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/35 file after the snapshot 2023-10-30T12:48:21.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/36 file after the snapshot 2023-10-30T12:48:21.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/37 file after the snapshot 2023-10-30T12:48:21.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/38 file after the snapshot 2023-10-30T12:48:21.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 244/39 file after the snapshot 2023-10-30T12:48:21.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/.snap/snap-subdir-test 2023-10-30T12:48:21.594 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/30 file after the snapshot 2023-10-30T12:48:21.595 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/31 file after the snapshot 2023-10-30T12:48:21.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/32 file after the snapshot 2023-10-30T12:48:21.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/33 file after the snapshot 2023-10-30T12:48:21.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/34 file after the snapshot 2023-10-30T12:48:21.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/35 file after the snapshot 2023-10-30T12:48:21.596 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/36 file after the snapshot 2023-10-30T12:48:21.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/37 file after the snapshot 2023-10-30T12:48:21.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/38 file after the snapshot 2023-10-30T12:48:21.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 245/39 file after the snapshot 2023-10-30T12:48:21.597 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/.snap/snap-subdir-test 2023-10-30T12:48:21.619 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/30 file after the snapshot 2023-10-30T12:48:21.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/31 file after the snapshot 2023-10-30T12:48:21.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/32 file after the snapshot 2023-10-30T12:48:21.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/33 file after the snapshot 2023-10-30T12:48:21.622 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/34 file after the snapshot 2023-10-30T12:48:21.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/35 file after the snapshot 2023-10-30T12:48:21.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/36 file after the snapshot 2023-10-30T12:48:21.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/37 file after the snapshot 2023-10-30T12:48:21.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/38 file after the snapshot 2023-10-30T12:48:21.623 INFO:tasks.workunit.client.0.smithi008.stdout: create 246/39 file after the snapshot 2023-10-30T12:48:21.624 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/.snap/snap-subdir-test 2023-10-30T12:48:21.646 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/30 file after the snapshot 2023-10-30T12:48:21.648 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/31 file after the snapshot 2023-10-30T12:48:21.648 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/32 file after the snapshot 2023-10-30T12:48:21.648 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/33 file after the snapshot 2023-10-30T12:48:21.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/34 file after the snapshot 2023-10-30T12:48:21.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/35 file after the snapshot 2023-10-30T12:48:21.649 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/36 file after the snapshot 2023-10-30T12:48:21.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/37 file after the snapshot 2023-10-30T12:48:21.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/38 file after the snapshot 2023-10-30T12:48:21.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 247/39 file after the snapshot 2023-10-30T12:48:21.650 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/.snap/snap-subdir-test 2023-10-30T12:48:21.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/30 file after the snapshot 2023-10-30T12:48:21.679 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/31 file after the snapshot 2023-10-30T12:48:21.679 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/32 file after the snapshot 2023-10-30T12:48:21.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/33 file after the snapshot 2023-10-30T12:48:21.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/34 file after the snapshot 2023-10-30T12:48:21.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/35 file after the snapshot 2023-10-30T12:48:21.680 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/36 file after the snapshot 2023-10-30T12:48:21.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/37 file after the snapshot 2023-10-30T12:48:21.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/38 file after the snapshot 2023-10-30T12:48:21.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 248/39 file after the snapshot 2023-10-30T12:48:21.681 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/.snap/snap-subdir-test 2023-10-30T12:48:21.708 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/30 file after the snapshot 2023-10-30T12:48:21.709 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/31 file after the snapshot 2023-10-30T12:48:21.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/32 file after the snapshot 2023-10-30T12:48:21.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/33 file after the snapshot 2023-10-30T12:48:21.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/34 file after the snapshot 2023-10-30T12:48:21.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/35 file after the snapshot 2023-10-30T12:48:21.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/36 file after the snapshot 2023-10-30T12:48:21.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/37 file after the snapshot 2023-10-30T12:48:21.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/38 file after the snapshot 2023-10-30T12:48:21.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 249/39 file after the snapshot 2023-10-30T12:48:21.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/.snap/snap-subdir-test 2023-10-30T12:48:21.737 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/30 file after the snapshot 2023-10-30T12:48:21.739 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/31 file after the snapshot 2023-10-30T12:48:21.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/32 file after the snapshot 2023-10-30T12:48:21.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/33 file after the snapshot 2023-10-30T12:48:21.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/34 file after the snapshot 2023-10-30T12:48:21.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/35 file after the snapshot 2023-10-30T12:48:21.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/36 file after the snapshot 2023-10-30T12:48:21.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/37 file after the snapshot 2023-10-30T12:48:21.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/38 file after the snapshot 2023-10-30T12:48:21.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 250/39 file after the snapshot 2023-10-30T12:48:21.742 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/.snap/snap-subdir-test 2023-10-30T12:48:22.535 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/30 file after the snapshot 2023-10-30T12:48:22.538 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/31 file after the snapshot 2023-10-30T12:48:22.538 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/32 file after the snapshot 2023-10-30T12:48:22.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/33 file after the snapshot 2023-10-30T12:48:22.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/34 file after the snapshot 2023-10-30T12:48:22.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/35 file after the snapshot 2023-10-30T12:48:22.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/36 file after the snapshot 2023-10-30T12:48:22.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/37 file after the snapshot 2023-10-30T12:48:22.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/38 file after the snapshot 2023-10-30T12:48:22.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 251/39 file after the snapshot 2023-10-30T12:48:22.540 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/.snap/snap-subdir-test 2023-10-30T12:48:24.123 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/30 file after the snapshot 2023-10-30T12:48:24.124 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/31 file after the snapshot 2023-10-30T12:48:24.124 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/32 file after the snapshot 2023-10-30T12:48:24.125 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/33 file after the snapshot 2023-10-30T12:48:24.125 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/34 file after the snapshot 2023-10-30T12:48:24.125 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/35 file after the snapshot 2023-10-30T12:48:24.126 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/36 file after the snapshot 2023-10-30T12:48:24.126 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/37 file after the snapshot 2023-10-30T12:48:24.126 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/38 file after the snapshot 2023-10-30T12:48:24.126 INFO:tasks.workunit.client.0.smithi008.stdout: create 252/39 file after the snapshot 2023-10-30T12:48:24.127 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/.snap/snap-subdir-test 2023-10-30T12:48:24.148 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/30 file after the snapshot 2023-10-30T12:48:24.149 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/31 file after the snapshot 2023-10-30T12:48:24.149 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/32 file after the snapshot 2023-10-30T12:48:24.150 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/33 file after the snapshot 2023-10-30T12:48:24.150 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/34 file after the snapshot 2023-10-30T12:48:24.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/35 file after the snapshot 2023-10-30T12:48:24.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/36 file after the snapshot 2023-10-30T12:48:24.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/37 file after the snapshot 2023-10-30T12:48:24.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/38 file after the snapshot 2023-10-30T12:48:24.151 INFO:tasks.workunit.client.0.smithi008.stdout: create 253/39 file after the snapshot 2023-10-30T12:48:24.152 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/.snap/snap-subdir-test 2023-10-30T12:48:24.182 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/30 file after the snapshot 2023-10-30T12:48:24.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/31 file after the snapshot 2023-10-30T12:48:24.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/32 file after the snapshot 2023-10-30T12:48:24.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/33 file after the snapshot 2023-10-30T12:48:24.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/34 file after the snapshot 2023-10-30T12:48:24.184 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/35 file after the snapshot 2023-10-30T12:48:24.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/36 file after the snapshot 2023-10-30T12:48:24.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/37 file after the snapshot 2023-10-30T12:48:24.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/38 file after the snapshot 2023-10-30T12:48:24.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 254/39 file after the snapshot 2023-10-30T12:48:24.185 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/.snap/snap-subdir-test 2023-10-30T12:48:24.210 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/30 file after the snapshot 2023-10-30T12:48:24.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/31 file after the snapshot 2023-10-30T12:48:24.211 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/32 file after the snapshot 2023-10-30T12:48:24.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/33 file after the snapshot 2023-10-30T12:48:24.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/34 file after the snapshot 2023-10-30T12:48:24.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/35 file after the snapshot 2023-10-30T12:48:24.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/36 file after the snapshot 2023-10-30T12:48:24.212 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/37 file after the snapshot 2023-10-30T12:48:24.213 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/38 file after the snapshot 2023-10-30T12:48:24.213 INFO:tasks.workunit.client.0.smithi008.stdout: create 255/39 file after the snapshot 2023-10-30T12:48:24.213 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/.snap/snap-subdir-test 2023-10-30T12:48:24.228 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/30 file after the snapshot 2023-10-30T12:48:24.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/31 file after the snapshot 2023-10-30T12:48:24.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/32 file after the snapshot 2023-10-30T12:48:24.231 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/33 file after the snapshot 2023-10-30T12:48:24.232 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/34 file after the snapshot 2023-10-30T12:48:24.232 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/35 file after the snapshot 2023-10-30T12:48:24.233 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/36 file after the snapshot 2023-10-30T12:48:24.233 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/37 file after the snapshot 2023-10-30T12:48:24.233 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/38 file after the snapshot 2023-10-30T12:48:24.234 INFO:tasks.workunit.client.0.smithi008.stdout: create 256/39 file after the snapshot 2023-10-30T12:48:24.234 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/.snap/snap-subdir-test 2023-10-30T12:48:24.257 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/30 file after the snapshot 2023-10-30T12:48:24.258 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/31 file after the snapshot 2023-10-30T12:48:24.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/32 file after the snapshot 2023-10-30T12:48:24.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/33 file after the snapshot 2023-10-30T12:48:24.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/34 file after the snapshot 2023-10-30T12:48:24.259 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/35 file after the snapshot 2023-10-30T12:48:24.260 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/36 file after the snapshot 2023-10-30T12:48:24.260 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/37 file after the snapshot 2023-10-30T12:48:24.260 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/38 file after the snapshot 2023-10-30T12:48:24.260 INFO:tasks.workunit.client.0.smithi008.stdout: create 257/39 file after the snapshot 2023-10-30T12:48:24.260 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/.snap/snap-subdir-test 2023-10-30T12:48:24.276 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/30 file after the snapshot 2023-10-30T12:48:24.528 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/31 file after the snapshot 2023-10-30T12:48:24.529 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/32 file after the snapshot 2023-10-30T12:48:24.529 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/33 file after the snapshot 2023-10-30T12:48:24.529 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/34 file after the snapshot 2023-10-30T12:48:24.529 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/35 file after the snapshot 2023-10-30T12:48:24.530 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/36 file after the snapshot 2023-10-30T12:48:24.530 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/37 file after the snapshot 2023-10-30T12:48:24.530 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/38 file after the snapshot 2023-10-30T12:48:24.530 INFO:tasks.workunit.client.0.smithi008.stdout: create 258/39 file after the snapshot 2023-10-30T12:48:24.530 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/.snap/snap-subdir-test 2023-10-30T12:48:24.552 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/30 file after the snapshot 2023-10-30T12:48:24.554 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/31 file after the snapshot 2023-10-30T12:48:24.554 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/32 file after the snapshot 2023-10-30T12:48:24.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/33 file after the snapshot 2023-10-30T12:48:24.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/34 file after the snapshot 2023-10-30T12:48:24.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/35 file after the snapshot 2023-10-30T12:48:24.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/36 file after the snapshot 2023-10-30T12:48:24.555 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/37 file after the snapshot 2023-10-30T12:48:24.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/38 file after the snapshot 2023-10-30T12:48:24.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 259/39 file after the snapshot 2023-10-30T12:48:24.556 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/.snap/snap-subdir-test 2023-10-30T12:48:24.711 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/30 file after the snapshot 2023-10-30T12:48:24.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/31 file after the snapshot 2023-10-30T12:48:24.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/32 file after the snapshot 2023-10-30T12:48:24.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/33 file after the snapshot 2023-10-30T12:48:24.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/34 file after the snapshot 2023-10-30T12:48:24.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/35 file after the snapshot 2023-10-30T12:48:24.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/36 file after the snapshot 2023-10-30T12:48:24.715 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/37 file after the snapshot 2023-10-30T12:48:24.716 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/38 file after the snapshot 2023-10-30T12:48:24.716 INFO:tasks.workunit.client.0.smithi008.stdout: create 260/39 file after the snapshot 2023-10-30T12:48:24.716 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/.snap/snap-subdir-test 2023-10-30T12:48:24.738 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/30 file after the snapshot 2023-10-30T12:48:24.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/31 file after the snapshot 2023-10-30T12:48:24.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/32 file after the snapshot 2023-10-30T12:48:24.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/33 file after the snapshot 2023-10-30T12:48:24.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/34 file after the snapshot 2023-10-30T12:48:24.740 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/35 file after the snapshot 2023-10-30T12:48:24.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/36 file after the snapshot 2023-10-30T12:48:24.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/37 file after the snapshot 2023-10-30T12:48:24.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/38 file after the snapshot 2023-10-30T12:48:24.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 261/39 file after the snapshot 2023-10-30T12:48:24.741 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/.snap/snap-subdir-test 2023-10-30T12:48:25.095 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/30 file after the snapshot 2023-10-30T12:48:25.097 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/31 file after the snapshot 2023-10-30T12:48:25.097 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/32 file after the snapshot 2023-10-30T12:48:25.097 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/33 file after the snapshot 2023-10-30T12:48:25.097 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/34 file after the snapshot 2023-10-30T12:48:25.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/35 file after the snapshot 2023-10-30T12:48:25.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/36 file after the snapshot 2023-10-30T12:48:25.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/37 file after the snapshot 2023-10-30T12:48:25.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/38 file after the snapshot 2023-10-30T12:48:25.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 262/39 file after the snapshot 2023-10-30T12:48:25.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/.snap/snap-subdir-test 2023-10-30T12:48:25.139 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/30 file after the snapshot 2023-10-30T12:48:25.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/31 file after the snapshot 2023-10-30T12:48:25.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/32 file after the snapshot 2023-10-30T12:48:25.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/33 file after the snapshot 2023-10-30T12:48:25.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/34 file after the snapshot 2023-10-30T12:48:25.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/35 file after the snapshot 2023-10-30T12:48:25.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/36 file after the snapshot 2023-10-30T12:48:25.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/37 file after the snapshot 2023-10-30T12:48:25.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/38 file after the snapshot 2023-10-30T12:48:25.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 263/39 file after the snapshot 2023-10-30T12:48:25.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/.snap/snap-subdir-test 2023-10-30T12:48:25.162 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/30 file after the snapshot 2023-10-30T12:48:25.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/31 file after the snapshot 2023-10-30T12:48:25.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/32 file after the snapshot 2023-10-30T12:48:25.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/33 file after the snapshot 2023-10-30T12:48:25.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/34 file after the snapshot 2023-10-30T12:48:25.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/35 file after the snapshot 2023-10-30T12:48:25.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/36 file after the snapshot 2023-10-30T12:48:25.165 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/37 file after the snapshot 2023-10-30T12:48:25.166 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/38 file after the snapshot 2023-10-30T12:48:25.166 INFO:tasks.workunit.client.0.smithi008.stdout: create 264/39 file after the snapshot 2023-10-30T12:48:25.166 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/.snap/snap-subdir-test 2023-10-30T12:48:25.197 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/30 file after the snapshot 2023-10-30T12:48:25.198 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/31 file after the snapshot 2023-10-30T12:48:25.199 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/32 file after the snapshot 2023-10-30T12:48:25.200 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/33 file after the snapshot 2023-10-30T12:48:25.201 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/34 file after the snapshot 2023-10-30T12:48:25.201 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/35 file after the snapshot 2023-10-30T12:48:25.202 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/36 file after the snapshot 2023-10-30T12:48:25.203 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/37 file after the snapshot 2023-10-30T12:48:25.203 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/38 file after the snapshot 2023-10-30T12:48:25.204 INFO:tasks.workunit.client.0.smithi008.stdout: create 265/39 file after the snapshot 2023-10-30T12:48:25.205 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/.snap/snap-subdir-test 2023-10-30T12:48:25.223 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/30 file after the snapshot 2023-10-30T12:48:25.224 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/31 file after the snapshot 2023-10-30T12:48:25.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/32 file after the snapshot 2023-10-30T12:48:25.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/33 file after the snapshot 2023-10-30T12:48:25.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/34 file after the snapshot 2023-10-30T12:48:25.225 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/35 file after the snapshot 2023-10-30T12:48:25.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/36 file after the snapshot 2023-10-30T12:48:25.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/37 file after the snapshot 2023-10-30T12:48:25.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/38 file after the snapshot 2023-10-30T12:48:25.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 266/39 file after the snapshot 2023-10-30T12:48:25.226 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/.snap/snap-subdir-test 2023-10-30T12:48:25.243 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/30 file after the snapshot 2023-10-30T12:48:25.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/31 file after the snapshot 2023-10-30T12:48:25.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/32 file after the snapshot 2023-10-30T12:48:25.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/33 file after the snapshot 2023-10-30T12:48:25.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/34 file after the snapshot 2023-10-30T12:48:25.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/35 file after the snapshot 2023-10-30T12:48:25.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/36 file after the snapshot 2023-10-30T12:48:25.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/37 file after the snapshot 2023-10-30T12:48:25.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/38 file after the snapshot 2023-10-30T12:48:25.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 267/39 file after the snapshot 2023-10-30T12:48:25.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/.snap/snap-subdir-test 2023-10-30T12:48:25.826 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/30 file after the snapshot 2023-10-30T12:48:25.827 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/31 file after the snapshot 2023-10-30T12:48:25.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/32 file after the snapshot 2023-10-30T12:48:25.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/33 file after the snapshot 2023-10-30T12:48:25.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/34 file after the snapshot 2023-10-30T12:48:25.828 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/35 file after the snapshot 2023-10-30T12:48:25.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/36 file after the snapshot 2023-10-30T12:48:25.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/37 file after the snapshot 2023-10-30T12:48:25.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/38 file after the snapshot 2023-10-30T12:48:25.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 268/39 file after the snapshot 2023-10-30T12:48:25.829 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/.snap/snap-subdir-test 2023-10-30T12:48:25.853 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/30 file after the snapshot 2023-10-30T12:48:25.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/31 file after the snapshot 2023-10-30T12:48:25.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/32 file after the snapshot 2023-10-30T12:48:25.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/33 file after the snapshot 2023-10-30T12:48:25.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/34 file after the snapshot 2023-10-30T12:48:25.855 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/35 file after the snapshot 2023-10-30T12:48:25.856 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/36 file after the snapshot 2023-10-30T12:48:25.856 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/37 file after the snapshot 2023-10-30T12:48:25.856 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/38 file after the snapshot 2023-10-30T12:48:25.856 INFO:tasks.workunit.client.0.smithi008.stdout: create 269/39 file after the snapshot 2023-10-30T12:48:25.856 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/.snap/snap-subdir-test 2023-10-30T12:48:25.889 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/30 file after the snapshot 2023-10-30T12:48:25.892 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/31 file after the snapshot 2023-10-30T12:48:25.892 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/32 file after the snapshot 2023-10-30T12:48:25.892 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/33 file after the snapshot 2023-10-30T12:48:25.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/34 file after the snapshot 2023-10-30T12:48:25.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/35 file after the snapshot 2023-10-30T12:48:25.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/36 file after the snapshot 2023-10-30T12:48:25.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/37 file after the snapshot 2023-10-30T12:48:25.893 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/38 file after the snapshot 2023-10-30T12:48:25.894 INFO:tasks.workunit.client.0.smithi008.stdout: create 270/39 file after the snapshot 2023-10-30T12:48:25.894 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/.snap/snap-subdir-test 2023-10-30T12:48:26.581 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/30 file after the snapshot 2023-10-30T12:48:26.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/31 file after the snapshot 2023-10-30T12:48:26.584 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/32 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/33 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/34 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/35 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/36 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/37 file after the snapshot 2023-10-30T12:48:26.585 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/38 file after the snapshot 2023-10-30T12:48:26.586 INFO:tasks.workunit.client.0.smithi008.stdout: create 271/39 file after the snapshot 2023-10-30T12:48:26.586 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/.snap/snap-subdir-test 2023-10-30T12:48:26.603 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/30 file after the snapshot 2023-10-30T12:48:26.605 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/31 file after the snapshot 2023-10-30T12:48:26.605 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/32 file after the snapshot 2023-10-30T12:48:26.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/33 file after the snapshot 2023-10-30T12:48:26.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/34 file after the snapshot 2023-10-30T12:48:26.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/35 file after the snapshot 2023-10-30T12:48:26.606 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/36 file after the snapshot 2023-10-30T12:48:26.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/37 file after the snapshot 2023-10-30T12:48:26.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/38 file after the snapshot 2023-10-30T12:48:26.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 272/39 file after the snapshot 2023-10-30T12:48:26.607 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/.snap/snap-subdir-test 2023-10-30T12:48:26.632 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/30 file after the snapshot 2023-10-30T12:48:26.633 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/31 file after the snapshot 2023-10-30T12:48:26.634 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/32 file after the snapshot 2023-10-30T12:48:26.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/33 file after the snapshot 2023-10-30T12:48:26.635 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/34 file after the snapshot 2023-10-30T12:48:26.636 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/35 file after the snapshot 2023-10-30T12:48:26.637 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/36 file after the snapshot 2023-10-30T12:48:26.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/37 file after the snapshot 2023-10-30T12:48:26.638 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/38 file after the snapshot 2023-10-30T12:48:26.639 INFO:tasks.workunit.client.0.smithi008.stdout: create 273/39 file after the snapshot 2023-10-30T12:48:26.640 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/.snap/snap-subdir-test 2023-10-30T12:48:26.655 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/30 file after the snapshot 2023-10-30T12:48:26.655 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/31 file after the snapshot 2023-10-30T12:48:26.656 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/32 file after the snapshot 2023-10-30T12:48:26.657 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/33 file after the snapshot 2023-10-30T12:48:26.658 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/34 file after the snapshot 2023-10-30T12:48:26.658 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/35 file after the snapshot 2023-10-30T12:48:26.659 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/36 file after the snapshot 2023-10-30T12:48:26.660 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/37 file after the snapshot 2023-10-30T12:48:26.660 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/38 file after the snapshot 2023-10-30T12:48:26.661 INFO:tasks.workunit.client.0.smithi008.stdout: create 274/39 file after the snapshot 2023-10-30T12:48:26.662 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/.snap/snap-subdir-test 2023-10-30T12:48:26.675 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/30 file after the snapshot 2023-10-30T12:48:26.677 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/31 file after the snapshot 2023-10-30T12:48:26.677 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/32 file after the snapshot 2023-10-30T12:48:26.677 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/33 file after the snapshot 2023-10-30T12:48:26.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/34 file after the snapshot 2023-10-30T12:48:26.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/35 file after the snapshot 2023-10-30T12:48:26.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/36 file after the snapshot 2023-10-30T12:48:26.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/37 file after the snapshot 2023-10-30T12:48:26.678 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/38 file after the snapshot 2023-10-30T12:48:26.679 INFO:tasks.workunit.client.0.smithi008.stdout: create 275/39 file after the snapshot 2023-10-30T12:48:26.679 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/.snap/snap-subdir-test 2023-10-30T12:48:26.710 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/30 file after the snapshot 2023-10-30T12:48:26.712 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/31 file after the snapshot 2023-10-30T12:48:26.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/32 file after the snapshot 2023-10-30T12:48:26.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/33 file after the snapshot 2023-10-30T12:48:26.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/34 file after the snapshot 2023-10-30T12:48:26.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/35 file after the snapshot 2023-10-30T12:48:26.713 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/36 file after the snapshot 2023-10-30T12:48:26.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/37 file after the snapshot 2023-10-30T12:48:26.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/38 file after the snapshot 2023-10-30T12:48:26.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 276/39 file after the snapshot 2023-10-30T12:48:26.714 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/.snap/snap-subdir-test 2023-10-30T12:48:26.730 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/30 file after the snapshot 2023-10-30T12:48:26.732 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/31 file after the snapshot 2023-10-30T12:48:26.732 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/32 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/33 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/34 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/35 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/36 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/37 file after the snapshot 2023-10-30T12:48:26.733 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/38 file after the snapshot 2023-10-30T12:48:26.734 INFO:tasks.workunit.client.0.smithi008.stdout: create 277/39 file after the snapshot 2023-10-30T12:48:26.734 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/.snap/snap-subdir-test 2023-10-30T12:48:26.758 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/30 file after the snapshot 2023-10-30T12:48:26.760 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/31 file after the snapshot 2023-10-30T12:48:26.760 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/32 file after the snapshot 2023-10-30T12:48:26.760 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/33 file after the snapshot 2023-10-30T12:48:26.760 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/34 file after the snapshot 2023-10-30T12:48:26.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/35 file after the snapshot 2023-10-30T12:48:26.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/36 file after the snapshot 2023-10-30T12:48:26.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/37 file after the snapshot 2023-10-30T12:48:26.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/38 file after the snapshot 2023-10-30T12:48:26.761 INFO:tasks.workunit.client.0.smithi008.stdout: create 278/39 file after the snapshot 2023-10-30T12:48:26.762 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/.snap/snap-subdir-test 2023-10-30T12:48:26.985 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/30 file after the snapshot 2023-10-30T12:48:26.987 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/31 file after the snapshot 2023-10-30T12:48:26.987 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/32 file after the snapshot 2023-10-30T12:48:26.987 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/33 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/34 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/35 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/36 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/37 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/38 file after the snapshot 2023-10-30T12:48:26.988 INFO:tasks.workunit.client.0.smithi008.stdout: create 279/39 file after the snapshot 2023-10-30T12:48:26.989 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/.snap/snap-subdir-test 2023-10-30T12:48:27.009 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/30 file after the snapshot 2023-10-30T12:48:27.011 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/31 file after the snapshot 2023-10-30T12:48:27.011 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/32 file after the snapshot 2023-10-30T12:48:27.012 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/33 file after the snapshot 2023-10-30T12:48:27.012 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/34 file after the snapshot 2023-10-30T12:48:27.012 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/35 file after the snapshot 2023-10-30T12:48:27.012 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/36 file after the snapshot 2023-10-30T12:48:27.012 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/37 file after the snapshot 2023-10-30T12:48:27.013 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/38 file after the snapshot 2023-10-30T12:48:27.013 INFO:tasks.workunit.client.0.smithi008.stdout: create 280/39 file after the snapshot 2023-10-30T12:48:27.013 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/.snap/snap-subdir-test 2023-10-30T12:48:27.036 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/30 file after the snapshot 2023-10-30T12:48:27.038 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/31 file after the snapshot 2023-10-30T12:48:27.038 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/32 file after the snapshot 2023-10-30T12:48:27.038 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/33 file after the snapshot 2023-10-30T12:48:27.039 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/34 file after the snapshot 2023-10-30T12:48:27.039 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/35 file after the snapshot 2023-10-30T12:48:27.039 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/36 file after the snapshot 2023-10-30T12:48:27.039 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/37 file after the snapshot 2023-10-30T12:48:27.039 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/38 file after the snapshot 2023-10-30T12:48:27.040 INFO:tasks.workunit.client.0.smithi008.stdout: create 281/39 file after the snapshot 2023-10-30T12:48:27.040 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/.snap/snap-subdir-test 2023-10-30T12:48:27.063 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/30 file after the snapshot 2023-10-30T12:48:27.064 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/31 file after the snapshot 2023-10-30T12:48:27.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/32 file after the snapshot 2023-10-30T12:48:27.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/33 file after the snapshot 2023-10-30T12:48:27.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/34 file after the snapshot 2023-10-30T12:48:27.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/35 file after the snapshot 2023-10-30T12:48:27.065 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/36 file after the snapshot 2023-10-30T12:48:27.066 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/37 file after the snapshot 2023-10-30T12:48:27.066 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/38 file after the snapshot 2023-10-30T12:48:27.066 INFO:tasks.workunit.client.0.smithi008.stdout: create 282/39 file after the snapshot 2023-10-30T12:48:27.066 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/.snap/snap-subdir-test 2023-10-30T12:48:27.094 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/30 file after the snapshot 2023-10-30T12:48:27.097 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/31 file after the snapshot 2023-10-30T12:48:27.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/32 file after the snapshot 2023-10-30T12:48:27.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/33 file after the snapshot 2023-10-30T12:48:27.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/34 file after the snapshot 2023-10-30T12:48:27.098 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/35 file after the snapshot 2023-10-30T12:48:27.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/36 file after the snapshot 2023-10-30T12:48:27.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/37 file after the snapshot 2023-10-30T12:48:27.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/38 file after the snapshot 2023-10-30T12:48:27.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 283/39 file after the snapshot 2023-10-30T12:48:27.099 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/.snap/snap-subdir-test 2023-10-30T12:48:27.363 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/30 file after the snapshot 2023-10-30T12:48:27.364 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/31 file after the snapshot 2023-10-30T12:48:27.364 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/32 file after the snapshot 2023-10-30T12:48:27.364 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/33 file after the snapshot 2023-10-30T12:48:27.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/34 file after the snapshot 2023-10-30T12:48:27.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/35 file after the snapshot 2023-10-30T12:48:27.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/36 file after the snapshot 2023-10-30T12:48:27.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/37 file after the snapshot 2023-10-30T12:48:27.365 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/38 file after the snapshot 2023-10-30T12:48:27.366 INFO:tasks.workunit.client.0.smithi008.stdout: create 284/39 file after the snapshot 2023-10-30T12:48:27.366 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/.snap/snap-subdir-test 2023-10-30T12:48:27.389 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/30 file after the snapshot 2023-10-30T12:48:27.390 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/31 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/32 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/33 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/34 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/35 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/36 file after the snapshot 2023-10-30T12:48:27.391 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/37 file after the snapshot 2023-10-30T12:48:27.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/38 file after the snapshot 2023-10-30T12:48:27.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 285/39 file after the snapshot 2023-10-30T12:48:27.392 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/.snap/snap-subdir-test 2023-10-30T12:48:27.422 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/30 file after the snapshot 2023-10-30T12:48:27.425 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/31 file after the snapshot 2023-10-30T12:48:27.425 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/32 file after the snapshot 2023-10-30T12:48:27.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/33 file after the snapshot 2023-10-30T12:48:27.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/34 file after the snapshot 2023-10-30T12:48:27.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/35 file after the snapshot 2023-10-30T12:48:27.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/36 file after the snapshot 2023-10-30T12:48:27.426 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/37 file after the snapshot 2023-10-30T12:48:27.427 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/38 file after the snapshot 2023-10-30T12:48:27.427 INFO:tasks.workunit.client.0.smithi008.stdout: create 286/39 file after the snapshot 2023-10-30T12:48:27.427 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/.snap/snap-subdir-test 2023-10-30T12:48:27.441 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/30 file after the snapshot 2023-10-30T12:48:27.442 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/31 file after the snapshot 2023-10-30T12:48:27.443 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/32 file after the snapshot 2023-10-30T12:48:27.443 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/33 file after the snapshot 2023-10-30T12:48:27.444 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/34 file after the snapshot 2023-10-30T12:48:27.445 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/35 file after the snapshot 2023-10-30T12:48:27.445 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/36 file after the snapshot 2023-10-30T12:48:27.446 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/37 file after the snapshot 2023-10-30T12:48:27.447 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/38 file after the snapshot 2023-10-30T12:48:27.448 INFO:tasks.workunit.client.0.smithi008.stdout: create 287/39 file after the snapshot 2023-10-30T12:48:27.448 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/.snap/snap-subdir-test 2023-10-30T12:48:27.463 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/30 file after the snapshot 2023-10-30T12:48:27.466 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/31 file after the snapshot 2023-10-30T12:48:27.466 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/32 file after the snapshot 2023-10-30T12:48:27.466 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/33 file after the snapshot 2023-10-30T12:48:27.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/34 file after the snapshot 2023-10-30T12:48:27.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/35 file after the snapshot 2023-10-30T12:48:27.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/36 file after the snapshot 2023-10-30T12:48:27.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/37 file after the snapshot 2023-10-30T12:48:27.467 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/38 file after the snapshot 2023-10-30T12:48:27.468 INFO:tasks.workunit.client.0.smithi008.stdout: create 288/39 file after the snapshot 2023-10-30T12:48:27.468 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/.snap/snap-subdir-test 2023-10-30T12:48:27.486 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/30 file after the snapshot 2023-10-30T12:48:27.487 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/31 file after the snapshot 2023-10-30T12:48:27.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/32 file after the snapshot 2023-10-30T12:48:27.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/33 file after the snapshot 2023-10-30T12:48:27.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/34 file after the snapshot 2023-10-30T12:48:27.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/35 file after the snapshot 2023-10-30T12:48:27.488 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/36 file after the snapshot 2023-10-30T12:48:27.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/37 file after the snapshot 2023-10-30T12:48:27.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/38 file after the snapshot 2023-10-30T12:48:27.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 289/39 file after the snapshot 2023-10-30T12:48:27.489 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/.snap/snap-subdir-test 2023-10-30T12:48:27.512 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/30 file after the snapshot 2023-10-30T12:48:27.513 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/31 file after the snapshot 2023-10-30T12:48:27.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/32 file after the snapshot 2023-10-30T12:48:27.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/33 file after the snapshot 2023-10-30T12:48:27.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/34 file after the snapshot 2023-10-30T12:48:27.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/35 file after the snapshot 2023-10-30T12:48:27.514 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/36 file after the snapshot 2023-10-30T12:48:27.515 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/37 file after the snapshot 2023-10-30T12:48:27.515 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/38 file after the snapshot 2023-10-30T12:48:27.515 INFO:tasks.workunit.client.0.smithi008.stdout: create 290/39 file after the snapshot 2023-10-30T12:48:27.515 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/.snap/snap-subdir-test 2023-10-30T12:48:27.539 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/30 file after the snapshot 2023-10-30T12:48:27.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/31 file after the snapshot 2023-10-30T12:48:27.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/32 file after the snapshot 2023-10-30T12:48:27.541 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/33 file after the snapshot 2023-10-30T12:48:27.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/34 file after the snapshot 2023-10-30T12:48:27.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/35 file after the snapshot 2023-10-30T12:48:27.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/36 file after the snapshot 2023-10-30T12:48:27.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/37 file after the snapshot 2023-10-30T12:48:27.542 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/38 file after the snapshot 2023-10-30T12:48:27.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 291/39 file after the snapshot 2023-10-30T12:48:27.543 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/.snap/snap-subdir-test 2023-10-30T12:48:27.563 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/30 file after the snapshot 2023-10-30T12:48:27.564 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/31 file after the snapshot 2023-10-30T12:48:27.564 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/32 file after the snapshot 2023-10-30T12:48:27.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/33 file after the snapshot 2023-10-30T12:48:27.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/34 file after the snapshot 2023-10-30T12:48:27.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/35 file after the snapshot 2023-10-30T12:48:27.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/36 file after the snapshot 2023-10-30T12:48:27.565 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/37 file after the snapshot 2023-10-30T12:48:27.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/38 file after the snapshot 2023-10-30T12:48:27.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 292/39 file after the snapshot 2023-10-30T12:48:27.566 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/.snap/snap-subdir-test 2023-10-30T12:48:29.244 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/30 file after the snapshot 2023-10-30T12:48:29.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/31 file after the snapshot 2023-10-30T12:48:29.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/32 file after the snapshot 2023-10-30T12:48:29.247 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/33 file after the snapshot 2023-10-30T12:48:29.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/34 file after the snapshot 2023-10-30T12:48:29.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/35 file after the snapshot 2023-10-30T12:48:29.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/36 file after the snapshot 2023-10-30T12:48:29.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/37 file after the snapshot 2023-10-30T12:48:29.248 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/38 file after the snapshot 2023-10-30T12:48:29.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 293/39 file after the snapshot 2023-10-30T12:48:29.249 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/.snap/snap-subdir-test 2023-10-30T12:48:29.292 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/30 file after the snapshot 2023-10-30T12:48:29.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/31 file after the snapshot 2023-10-30T12:48:29.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/32 file after the snapshot 2023-10-30T12:48:29.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/33 file after the snapshot 2023-10-30T12:48:29.298 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/34 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/35 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/36 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/37 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/38 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 294/39 file after the snapshot 2023-10-30T12:48:29.299 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/.snap/snap-subdir-test 2023-10-30T12:48:29.314 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/30 file after the snapshot 2023-10-30T12:48:29.316 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/31 file after the snapshot 2023-10-30T12:48:29.316 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/32 file after the snapshot 2023-10-30T12:48:29.317 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/33 file after the snapshot 2023-10-30T12:48:29.317 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/34 file after the snapshot 2023-10-30T12:48:29.317 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/35 file after the snapshot 2023-10-30T12:48:29.318 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/36 file after the snapshot 2023-10-30T12:48:29.318 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/37 file after the snapshot 2023-10-30T12:48:29.318 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/38 file after the snapshot 2023-10-30T12:48:29.318 INFO:tasks.workunit.client.0.smithi008.stdout: create 295/39 file after the snapshot 2023-10-30T12:48:29.318 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/.snap/snap-subdir-test 2023-10-30T12:48:29.345 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/30 file after the snapshot 2023-10-30T12:48:29.346 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/31 file after the snapshot 2023-10-30T12:48:29.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/32 file after the snapshot 2023-10-30T12:48:29.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/33 file after the snapshot 2023-10-30T12:48:29.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/34 file after the snapshot 2023-10-30T12:48:29.347 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/35 file after the snapshot 2023-10-30T12:48:29.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/36 file after the snapshot 2023-10-30T12:48:29.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/37 file after the snapshot 2023-10-30T12:48:29.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/38 file after the snapshot 2023-10-30T12:48:29.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 296/39 file after the snapshot 2023-10-30T12:48:29.348 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/.snap/snap-subdir-test 2023-10-30T12:48:29.380 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/30 file after the snapshot 2023-10-30T12:48:29.383 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/31 file after the snapshot 2023-10-30T12:48:29.383 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/32 file after the snapshot 2023-10-30T12:48:29.384 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/33 file after the snapshot 2023-10-30T12:48:29.384 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/34 file after the snapshot 2023-10-30T12:48:29.384 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/35 file after the snapshot 2023-10-30T12:48:29.384 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/36 file after the snapshot 2023-10-30T12:48:29.385 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/37 file after the snapshot 2023-10-30T12:48:29.385 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/38 file after the snapshot 2023-10-30T12:48:29.385 INFO:tasks.workunit.client.0.smithi008.stdout: create 297/39 file after the snapshot 2023-10-30T12:48:29.385 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/.snap/snap-subdir-test 2023-10-30T12:48:30.138 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/30 file after the snapshot 2023-10-30T12:48:30.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/31 file after the snapshot 2023-10-30T12:48:30.141 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/32 file after the snapshot 2023-10-30T12:48:30.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/33 file after the snapshot 2023-10-30T12:48:30.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/34 file after the snapshot 2023-10-30T12:48:30.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/35 file after the snapshot 2023-10-30T12:48:30.142 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/36 file after the snapshot 2023-10-30T12:48:30.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/37 file after the snapshot 2023-10-30T12:48:30.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/38 file after the snapshot 2023-10-30T12:48:30.143 INFO:tasks.workunit.client.0.smithi008.stdout: create 298/39 file after the snapshot 2023-10-30T12:48:30.144 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/.snap/snap-subdir-test 2023-10-30T12:48:30.161 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/30 file after the snapshot 2023-10-30T12:48:30.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/31 file after the snapshot 2023-10-30T12:48:30.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/32 file after the snapshot 2023-10-30T12:48:30.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/33 file after the snapshot 2023-10-30T12:48:30.163 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/34 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/35 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/36 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/37 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/38 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout: create 299/39 file after the snapshot 2023-10-30T12:48:30.164 INFO:tasks.workunit.client.0.smithi008.stdout:Take second snapshot .snap/test2 2023-10-30T12:48:30.456 INFO:tasks.workunit.client.0.smithi008.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2023-10-30T12:48:30.898 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2023-10-30T12:48:30.898 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2023-10-30T12:48:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2023-10-30T12:48:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2023-10-30T12:48:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2023-10-30T12:48:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2023-10-30T12:48:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2023-10-30T12:48:30.900 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2023-10-30T12:48:30.900 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2023-10-30T12:48:30.900 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2023-10-30T12:48:30.900 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2023-10-30T12:48:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2023-10-30T12:48:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2023-10-30T12:48:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2023-10-30T12:48:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2023-10-30T12:48:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2023-10-30T12:48:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2023-10-30T12:48:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2023-10-30T12:48:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2023-10-30T12:48:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2023-10-30T12:48:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2023-10-30T12:48:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2023-10-30T12:48:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2023-10-30T12:48:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2023-10-30T12:48:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2023-10-30T12:48:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2023-10-30T12:48:30.904 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2023-10-30T12:48:30.904 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2023-10-30T12:48:30.904 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2023-10-30T12:48:30.904 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2023-10-30T12:48:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2023-10-30T12:48:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2023-10-30T12:48:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2023-10-30T12:48:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2023-10-30T12:48:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2023-10-30T12:48:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2023-10-30T12:48:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2023-10-30T12:48:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2023-10-30T12:48:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2023-10-30T12:48:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2023-10-30T12:48:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2023-10-30T12:48:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2023-10-30T12:48:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2023-10-30T12:48:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2023-10-30T12:48:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2023-10-30T12:48:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2023-10-30T12:48:30.909 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2023-10-30T12:48:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2023-10-30T12:48:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2023-10-30T12:48:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2023-10-30T12:48:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2023-10-30T12:48:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2023-10-30T12:48:30.911 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2023-10-30T12:48:30.912 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2023-10-30T12:48:30.912 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2023-10-30T12:48:30.912 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2023-10-30T12:48:30.912 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2023-10-30T12:48:30.912 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2023-10-30T12:48:30.913 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2023-10-30T12:48:30.913 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2023-10-30T12:48:30.913 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2023-10-30T12:48:30.913 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2023-10-30T12:48:30.913 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2023-10-30T12:48:30.914 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2023-10-30T12:48:30.914 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2023-10-30T12:48:30.914 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2023-10-30T12:48:31.061 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2023-10-30T12:48:31.061 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2023-10-30T12:48:31.061 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2023-10-30T12:48:31.062 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2023-10-30T12:48:31.062 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2023-10-30T12:48:31.062 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2023-10-30T12:48:31.062 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2023-10-30T12:48:31.063 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2023-10-30T12:48:31.063 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2023-10-30T12:48:31.063 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2023-10-30T12:48:31.063 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2023-10-30T12:48:31.063 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2023-10-30T12:48:31.064 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2023-10-30T12:48:31.064 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2023-10-30T12:48:31.064 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2023-10-30T12:48:31.065 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2023-10-30T12:48:31.065 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2023-10-30T12:48:31.065 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2023-10-30T12:48:31.065 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2023-10-30T12:48:31.066 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2023-10-30T12:48:31.066 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2023-10-30T12:48:31.066 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2023-10-30T12:48:31.066 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2023-10-30T12:48:31.066 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2023-10-30T12:48:31.067 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2023-10-30T12:48:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2023-10-30T12:48:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2023-10-30T12:48:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2023-10-30T12:48:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2023-10-30T12:48:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2023-10-30T12:48:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2023-10-30T12:48:31.070 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2023-10-30T12:48:31.070 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2023-10-30T12:48:31.070 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2023-10-30T12:48:31.070 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2023-10-30T12:48:31.070 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2023-10-30T12:48:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2023-10-30T12:48:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2023-10-30T12:48:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2023-10-30T12:48:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2023-10-30T12:48:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2023-10-30T12:48:31.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2023-10-30T12:48:31.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2023-10-30T12:48:31.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2023-10-30T12:48:31.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2023-10-30T12:48:31.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2023-10-30T12:48:31.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2023-10-30T12:48:31.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2023-10-30T12:48:31.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2023-10-30T12:48:31.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2023-10-30T12:48:31.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2023-10-30T12:48:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2023-10-30T12:48:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2023-10-30T12:48:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2023-10-30T12:48:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2023-10-30T12:48:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2023-10-30T12:48:31.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2023-10-30T12:48:31.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2023-10-30T12:48:31.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2023-10-30T12:48:31.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2023-10-30T12:48:31.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2023-10-30T12:48:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2023-10-30T12:48:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2023-10-30T12:48:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2023-10-30T12:48:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2023-10-30T12:48:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2023-10-30T12:48:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2023-10-30T12:48:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2023-10-30T12:48:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2023-10-30T12:48:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2023-10-30T12:48:31.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2023-10-30T12:48:32.308 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2023-10-30T12:48:32.309 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2023-10-30T12:48:32.309 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2023-10-30T12:48:32.309 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2023-10-30T12:48:32.309 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2023-10-30T12:48:32.309 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2023-10-30T12:48:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2023-10-30T12:48:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2023-10-30T12:48:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2023-10-30T12:48:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2023-10-30T12:48:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2023-10-30T12:48:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2023-10-30T12:48:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2023-10-30T12:48:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2023-10-30T12:48:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2023-10-30T12:48:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2023-10-30T12:48:32.313 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2023-10-30T12:48:32.313 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2023-10-30T12:48:32.313 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2023-10-30T12:48:32.313 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2023-10-30T12:48:32.313 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2023-10-30T12:48:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2023-10-30T12:48:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2023-10-30T12:48:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2023-10-30T12:48:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2023-10-30T12:48:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2023-10-30T12:48:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2023-10-30T12:48:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2023-10-30T12:48:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2023-10-30T12:48:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2023-10-30T12:48:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2023-10-30T12:48:32.316 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2023-10-30T12:48:32.316 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2023-10-30T12:48:32.316 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2023-10-30T12:48:32.316 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2023-10-30T12:48:32.316 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2023-10-30T12:48:32.317 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2023-10-30T12:48:32.317 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2023-10-30T12:48:32.317 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2023-10-30T12:48:32.317 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2023-10-30T12:48:32.317 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2023-10-30T12:48:32.318 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2023-10-30T12:48:32.318 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2023-10-30T12:48:32.318 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2023-10-30T12:48:32.318 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2023-10-30T12:48:32.318 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2023-10-30T12:48:32.319 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2023-10-30T12:48:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2023-10-30T12:48:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2023-10-30T12:48:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2023-10-30T12:48:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2023-10-30T12:48:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2023-10-30T12:48:32.321 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2023-10-30T12:48:32.322 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2023-10-30T12:48:32.322 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2023-10-30T12:48:32.322 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2023-10-30T12:48:32.322 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2023-10-30T12:48:32.322 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2023-10-30T12:48:32.323 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2023-10-30T12:48:32.323 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2023-10-30T12:48:32.323 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2023-10-30T12:48:32.323 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2023-10-30T12:48:32.323 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2023-10-30T12:48:32.324 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2023-10-30T12:48:32.324 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2023-10-30T12:48:32.324 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2023-10-30T12:48:32.725 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2023-10-30T12:48:32.725 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2023-10-30T12:48:32.726 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2023-10-30T12:48:32.726 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2023-10-30T12:48:32.726 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2023-10-30T12:48:32.726 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2023-10-30T12:48:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2023-10-30T12:48:32.728 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2023-10-30T12:48:32.728 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2023-10-30T12:48:32.728 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2023-10-30T12:48:32.728 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2023-10-30T12:48:32.728 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2023-10-30T12:48:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2023-10-30T12:48:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2023-10-30T12:48:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2023-10-30T12:48:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2023-10-30T12:48:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2023-10-30T12:48:32.730 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2023-10-30T12:48:32.730 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2023-10-30T12:48:32.730 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2023-10-30T12:48:32.730 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2023-10-30T12:48:32.730 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2023-10-30T12:48:32.731 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2023-10-30T12:48:32.732 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2023-10-30T12:48:32.732 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2023-10-30T12:48:32.732 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2023-10-30T12:48:32.732 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2023-10-30T12:48:32.732 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2023-10-30T12:48:32.733 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2023-10-30T12:48:32.733 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2023-10-30T12:48:32.733 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2023-10-30T12:48:32.733 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2023-10-30T12:48:32.733 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2023-10-30T12:48:32.734 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2023-10-30T12:48:32.734 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2023-10-30T12:48:32.734 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2023-10-30T12:48:32.734 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2023-10-30T12:48:32.734 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2023-10-30T12:48:32.735 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2023-10-30T12:48:32.735 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2023-10-30T12:48:32.735 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2023-10-30T12:48:32.735 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2023-10-30T12:48:32.735 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2023-10-30T12:48:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2023-10-30T12:48:32.737 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2023-10-30T12:48:32.737 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2023-10-30T12:48:32.737 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2023-10-30T12:48:32.737 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2023-10-30T12:48:32.737 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2023-10-30T12:48:32.738 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2023-10-30T12:48:32.738 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2023-10-30T12:48:32.738 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2023-10-30T12:48:32.738 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2023-10-30T12:48:32.738 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2023-10-30T12:48:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2023-10-30T12:48:32.740 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2023-10-30T12:48:32.740 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2023-10-30T12:48:32.740 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2023-10-30T12:48:32.740 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2023-10-30T12:48:32.740 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2023-10-30T12:48:32.741 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2023-10-30T12:48:34.088 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2023-10-30T12:48:34.088 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2023-10-30T12:48:34.088 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2023-10-30T12:48:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2023-10-30T12:48:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2023-10-30T12:48:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2023-10-30T12:48:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2023-10-30T12:48:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2023-10-30T12:48:34.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2023-10-30T12:48:34.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2023-10-30T12:48:34.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2023-10-30T12:48:34.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2023-10-30T12:48:34.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2023-10-30T12:48:34.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2023-10-30T12:48:34.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2023-10-30T12:48:34.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2023-10-30T12:48:34.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2023-10-30T12:48:34.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2023-10-30T12:48:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2023-10-30T12:48:34.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2023-10-30T12:48:34.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2023-10-30T12:48:34.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2023-10-30T12:48:34.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2023-10-30T12:48:34.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2023-10-30T12:48:34.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2023-10-30T12:48:34.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2023-10-30T12:48:34.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2023-10-30T12:48:34.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2023-10-30T12:48:34.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2023-10-30T12:48:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2023-10-30T12:48:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2023-10-30T12:48:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2023-10-30T12:48:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2023-10-30T12:48:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2023-10-30T12:48:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2023-10-30T12:48:34.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2023-10-30T12:48:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2023-10-30T12:48:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2023-10-30T12:48:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2023-10-30T12:48:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2023-10-30T12:48:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2023-10-30T12:48:34.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2023-10-30T12:48:34.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2023-10-30T12:48:34.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2023-10-30T12:48:34.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2023-10-30T12:48:34.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2023-10-30T12:48:34.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2023-10-30T12:48:34.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2023-10-30T12:48:34.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2023-10-30T12:48:34.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2023-10-30T12:48:34.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2023-10-30T12:48:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2023-10-30T12:48:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2023-10-30T12:48:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2023-10-30T12:48:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2023-10-30T12:48:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2023-10-30T12:48:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2023-10-30T12:48:34.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2023-10-30T12:48:34.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2023-10-30T12:48:34.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2023-10-30T12:48:34.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2023-10-30T12:48:37.088 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2023-10-30T12:48:37.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2023-10-30T12:48:37.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2023-10-30T12:48:37.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2023-10-30T12:48:37.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2023-10-30T12:48:37.089 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2023-10-30T12:48:37.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2023-10-30T12:48:37.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2023-10-30T12:48:37.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2023-10-30T12:48:37.090 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2023-10-30T12:48:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2023-10-30T12:48:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2023-10-30T12:48:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2023-10-30T12:48:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2023-10-30T12:48:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2023-10-30T12:48:37.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2023-10-30T12:48:37.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2023-10-30T12:48:37.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2023-10-30T12:48:37.092 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2023-10-30T12:48:37.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2023-10-30T12:48:37.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2023-10-30T12:48:37.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2023-10-30T12:48:37.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2023-10-30T12:48:37.093 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2023-10-30T12:48:37.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2023-10-30T12:48:37.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2023-10-30T12:48:37.094 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2023-10-30T12:48:37.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2023-10-30T12:48:37.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2023-10-30T12:48:37.095 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2023-10-30T12:48:37.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2023-10-30T12:48:37.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2023-10-30T12:48:37.096 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2023-10-30T12:48:37.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2023-10-30T12:48:37.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2023-10-30T12:48:37.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2023-10-30T12:48:37.097 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2023-10-30T12:48:37.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2023-10-30T12:48:37.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2023-10-30T12:48:37.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2023-10-30T12:48:37.098 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2023-10-30T12:48:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2023-10-30T12:48:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2023-10-30T12:48:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2023-10-30T12:48:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2023-10-30T12:48:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2023-10-30T12:48:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2023-10-30T12:48:37.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2023-10-30T12:48:37.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2023-10-30T12:48:37.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2023-10-30T12:48:37.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2023-10-30T12:48:37.101 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2023-10-30T12:48:37.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2023-10-30T12:48:37.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2023-10-30T12:48:37.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2023-10-30T12:48:37.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2023-10-30T12:48:37.102 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2023-10-30T12:48:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2023-10-30T12:48:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2023-10-30T12:48:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2023-10-30T12:48:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2023-10-30T12:48:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2023-10-30T12:48:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2023-10-30T12:48:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2023-10-30T12:48:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2023-10-30T12:48:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2023-10-30T12:48:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2023-10-30T12:48:37.105 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2023-10-30T12:48:37.106 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2023-10-30T12:48:37.106 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2023-10-30T12:48:37.106 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2023-10-30T12:48:38.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2023-10-30T12:48:38.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2023-10-30T12:48:38.071 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2023-10-30T12:48:38.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2023-10-30T12:48:38.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2023-10-30T12:48:38.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2023-10-30T12:48:38.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2023-10-30T12:48:38.072 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2023-10-30T12:48:38.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2023-10-30T12:48:38.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2023-10-30T12:48:38.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2023-10-30T12:48:38.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2023-10-30T12:48:38.073 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2023-10-30T12:48:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2023-10-30T12:48:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2023-10-30T12:48:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2023-10-30T12:48:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2023-10-30T12:48:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2023-10-30T12:48:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2023-10-30T12:48:38.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2023-10-30T12:48:38.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2023-10-30T12:48:38.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2023-10-30T12:48:38.076 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2023-10-30T12:48:38.077 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2023-10-30T12:48:38.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2023-10-30T12:48:38.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2023-10-30T12:48:38.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2023-10-30T12:48:38.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2023-10-30T12:48:38.078 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2023-10-30T12:48:38.079 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2023-10-30T12:48:38.079 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2023-10-30T12:48:38.079 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2023-10-30T12:48:38.079 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2023-10-30T12:48:38.079 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2023-10-30T12:48:38.080 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2023-10-30T12:48:38.080 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2023-10-30T12:48:38.080 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2023-10-30T12:48:38.080 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2023-10-30T12:48:38.080 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2023-10-30T12:48:38.081 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2023-10-30T12:48:38.081 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2023-10-30T12:48:38.081 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2023-10-30T12:48:38.081 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2023-10-30T12:48:38.081 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2023-10-30T12:48:38.082 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2023-10-30T12:48:38.082 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2023-10-30T12:48:38.082 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2023-10-30T12:48:38.082 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2023-10-30T12:48:38.083 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2023-10-30T12:48:38.083 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2023-10-30T12:48:38.083 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2023-10-30T12:48:38.083 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2023-10-30T12:48:38.083 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2023-10-30T12:48:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2023-10-30T12:48:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2023-10-30T12:48:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2023-10-30T12:48:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2023-10-30T12:48:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2023-10-30T12:48:38.085 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2023-10-30T12:48:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2023-10-30T12:48:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2023-10-30T12:48:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2023-10-30T12:48:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2023-10-30T12:48:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2023-10-30T12:48:38.087 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2023-10-30T12:48:38.087 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2023-10-30T12:48:38.087 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2023-10-30T12:48:38.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2023-10-30T12:48:38.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2023-10-30T12:48:38.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2023-10-30T12:48:38.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2023-10-30T12:48:38.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2023-10-30T12:48:38.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2023-10-30T12:48:38.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2023-10-30T12:48:38.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2023-10-30T12:48:38.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2023-10-30T12:48:38.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2023-10-30T12:48:38.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2023-10-30T12:48:38.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2023-10-30T12:48:38.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2023-10-30T12:48:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2023-10-30T12:48:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2023-10-30T12:48:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2023-10-30T12:48:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2023-10-30T12:48:38.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2023-10-30T12:48:38.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2023-10-30T12:48:38.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2023-10-30T12:48:38.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2023-10-30T12:48:38.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2023-10-30T12:48:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2023-10-30T12:48:38.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2023-10-30T12:48:38.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2023-10-30T12:48:38.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2023-10-30T12:48:38.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2023-10-30T12:48:38.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2023-10-30T12:48:38.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2023-10-30T12:48:38.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2023-10-30T12:48:38.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2023-10-30T12:48:38.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2023-10-30T12:48:38.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2023-10-30T12:48:38.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2023-10-30T12:48:38.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2023-10-30T12:48:38.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2023-10-30T12:48:38.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2023-10-30T12:48:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2023-10-30T12:48:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2023-10-30T12:48:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2023-10-30T12:48:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2023-10-30T12:48:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2023-10-30T12:48:38.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2023-10-30T12:48:38.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2023-10-30T12:48:38.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2023-10-30T12:48:38.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2023-10-30T12:48:38.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2023-10-30T12:48:38.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2023-10-30T12:48:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2023-10-30T12:48:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2023-10-30T12:48:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2023-10-30T12:48:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2023-10-30T12:48:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2023-10-30T12:48:38.519 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2023-10-30T12:48:38.519 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2023-10-30T12:48:38.519 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2023-10-30T12:48:38.519 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2023-10-30T12:48:38.519 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2023-10-30T12:48:38.520 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2023-10-30T12:48:38.520 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2023-10-30T12:48:38.520 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2023-10-30T12:48:38.520 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2023-10-30T12:48:38.520 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2023-10-30T12:48:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2023-10-30T12:48:38.522 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2023-10-30T12:48:38.522 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2023-10-30T12:48:38.662 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2023-10-30T12:48:38.662 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2023-10-30T12:48:38.662 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2023-10-30T12:48:38.662 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2023-10-30T12:48:38.663 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2023-10-30T12:48:38.663 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2023-10-30T12:48:38.663 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2023-10-30T12:48:38.663 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2023-10-30T12:48:38.663 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2023-10-30T12:48:38.664 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2023-10-30T12:48:38.664 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2023-10-30T12:48:38.664 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2023-10-30T12:48:38.664 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2023-10-30T12:48:38.664 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2023-10-30T12:48:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2023-10-30T12:48:38.666 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2023-10-30T12:48:38.666 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2023-10-30T12:48:38.666 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2023-10-30T12:48:38.666 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2023-10-30T12:48:38.666 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2023-10-30T12:48:38.667 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2023-10-30T12:48:38.667 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2023-10-30T12:48:38.667 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2023-10-30T12:48:38.667 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2023-10-30T12:48:38.668 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2023-10-30T12:48:38.668 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2023-10-30T12:48:38.668 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2023-10-30T12:48:38.668 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2023-10-30T12:48:38.668 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2023-10-30T12:48:38.669 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2023-10-30T12:48:38.669 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2023-10-30T12:48:38.669 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2023-10-30T12:48:38.669 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2023-10-30T12:48:38.669 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2023-10-30T12:48:38.670 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2023-10-30T12:48:38.670 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2023-10-30T12:48:38.670 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2023-10-30T12:48:38.670 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2023-10-30T12:48:38.671 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2023-10-30T12:48:38.671 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2023-10-30T12:48:38.671 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2023-10-30T12:48:38.671 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2023-10-30T12:48:38.671 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2023-10-30T12:48:38.672 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2023-10-30T12:48:38.672 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2023-10-30T12:48:38.672 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2023-10-30T12:48:38.672 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2023-10-30T12:48:38.672 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2023-10-30T12:48:38.673 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2023-10-30T12:48:38.673 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2023-10-30T12:48:38.673 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2023-10-30T12:48:38.673 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2023-10-30T12:48:38.673 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2023-10-30T12:48:38.674 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2023-10-30T12:48:38.674 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2023-10-30T12:48:38.674 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2023-10-30T12:48:38.674 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2023-10-30T12:48:38.674 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2023-10-30T12:48:38.675 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2023-10-30T12:48:38.675 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2023-10-30T12:48:38.675 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2023-10-30T12:48:38.675 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2023-10-30T12:48:38.675 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2023-10-30T12:48:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2023-10-30T12:48:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2023-10-30T12:48:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2023-10-30T12:48:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2023-10-30T12:48:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2023-10-30T12:48:38.677 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2023-10-30T12:48:38.677 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2023-10-30T12:48:38.677 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2023-10-30T12:48:38.677 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2023-10-30T12:48:38.677 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2023-10-30T12:48:38.678 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2023-10-30T12:48:38.678 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2023-10-30T12:48:38.678 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2023-10-30T12:48:40.109 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2023-10-30T12:48:40.110 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2023-10-30T12:48:40.110 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2023-10-30T12:48:40.110 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2023-10-30T12:48:40.111 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2023-10-30T12:48:40.111 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2023-10-30T12:48:40.111 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2023-10-30T12:48:40.111 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2023-10-30T12:48:40.111 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2023-10-30T12:48:40.112 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2023-10-30T12:48:40.112 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2023-10-30T12:48:40.112 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2023-10-30T12:48:40.112 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2023-10-30T12:48:40.113 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2023-10-30T12:48:40.113 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2023-10-30T12:48:40.113 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2023-10-30T12:48:40.113 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2023-10-30T12:48:40.113 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2023-10-30T12:48:40.114 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2023-10-30T12:48:40.114 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2023-10-30T12:48:40.114 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2023-10-30T12:48:40.114 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2023-10-30T12:48:40.115 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2023-10-30T12:48:40.115 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2023-10-30T12:48:40.115 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2023-10-30T12:48:40.115 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2023-10-30T12:48:40.115 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2023-10-30T12:48:40.116 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2023-10-30T12:48:40.116 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2023-10-30T12:48:40.116 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2023-10-30T12:48:40.116 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2023-10-30T12:48:40.116 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2023-10-30T12:48:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2023-10-30T12:48:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2023-10-30T12:48:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2023-10-30T12:48:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2023-10-30T12:48:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2023-10-30T12:48:40.118 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2023-10-30T12:48:40.118 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2023-10-30T12:48:40.118 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2023-10-30T12:48:40.118 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2023-10-30T12:48:40.119 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2023-10-30T12:48:40.120 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2023-10-30T12:48:40.120 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2023-10-30T12:48:40.120 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2023-10-30T12:48:40.120 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2023-10-30T12:48:40.120 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2023-10-30T12:48:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2023-10-30T12:48:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2023-10-30T12:48:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2023-10-30T12:48:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2023-10-30T12:48:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2023-10-30T12:48:40.122 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2023-10-30T12:48:40.122 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2023-10-30T12:48:40.122 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2023-10-30T12:48:40.122 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2023-10-30T12:48:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2023-10-30T12:48:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2023-10-30T12:48:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2023-10-30T12:48:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2023-10-30T12:48:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2023-10-30T12:48:40.124 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2023-10-30T12:48:40.124 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2023-10-30T12:48:40.124 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2023-10-30T12:48:40.124 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2023-10-30T12:48:40.124 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2023-10-30T12:48:40.125 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2023-10-30T12:48:40.125 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2023-10-30T12:48:40.125 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2023-10-30T12:48:40.125 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2023-10-30T12:48:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2023-10-30T12:48:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2023-10-30T12:48:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2023-10-30T12:48:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2023-10-30T12:48:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2023-10-30T12:48:40.127 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2023-10-30T12:48:42.155 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2023-10-30T12:48:42.155 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2023-10-30T12:48:42.155 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2023-10-30T12:48:42.155 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2023-10-30T12:48:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2023-10-30T12:48:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2023-10-30T12:48:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2023-10-30T12:48:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2023-10-30T12:48:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2023-10-30T12:48:42.157 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2023-10-30T12:48:42.157 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2023-10-30T12:48:42.157 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2023-10-30T12:48:42.157 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2023-10-30T12:48:42.157 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2023-10-30T12:48:42.158 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2023-10-30T12:48:42.158 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2023-10-30T12:48:42.158 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2023-10-30T12:48:42.158 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2023-10-30T12:48:42.158 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2023-10-30T12:48:42.159 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2023-10-30T12:48:42.159 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2023-10-30T12:48:42.159 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2023-10-30T12:48:42.159 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2023-10-30T12:48:42.159 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2023-10-30T12:48:42.160 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2023-10-30T12:48:42.161 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2023-10-30T12:48:42.161 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2023-10-30T12:48:42.161 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2023-10-30T12:48:42.161 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2023-10-30T12:48:42.161 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2023-10-30T12:48:42.162 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2023-10-30T12:48:42.162 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2023-10-30T12:48:42.162 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2023-10-30T12:48:42.162 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2023-10-30T12:48:42.162 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2023-10-30T12:48:42.163 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2023-10-30T12:48:42.164 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2023-10-30T12:48:42.164 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2023-10-30T12:48:42.164 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2023-10-30T12:48:42.164 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2023-10-30T12:48:42.164 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2023-10-30T12:48:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2023-10-30T12:48:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2023-10-30T12:48:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2023-10-30T12:48:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2023-10-30T12:48:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2023-10-30T12:48:42.166 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2023-10-30T12:48:42.166 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2023-10-30T12:48:42.166 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2023-10-30T12:48:42.166 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2023-10-30T12:48:42.166 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2023-10-30T12:48:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2023-10-30T12:48:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2023-10-30T12:48:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2023-10-30T12:48:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2023-10-30T12:48:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2023-10-30T12:48:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2023-10-30T12:48:42.169 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2023-10-30T12:48:42.170 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2023-10-30T12:48:42.170 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2023-10-30T12:48:42.170 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2023-10-30T12:48:42.336 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2023-10-30T12:48:42.336 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2023-10-30T12:48:42.337 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2023-10-30T12:48:42.337 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2023-10-30T12:48:42.337 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2023-10-30T12:48:42.337 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2023-10-30T12:48:42.337 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2023-10-30T12:48:42.338 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2023-10-30T12:48:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2023-10-30T12:48:42.340 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2023-10-30T12:48:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2023-10-30T12:48:42.342 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2023-10-30T12:48:42.343 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2023-10-30T12:48:42.343 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2023-10-30T12:48:42.343 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2023-10-30T12:48:42.343 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2023-10-30T12:48:42.343 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2023-10-30T12:48:42.344 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2023-10-30T12:48:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2023-10-30T12:48:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2023-10-30T12:48:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2023-10-30T12:48:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2023-10-30T12:48:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2023-10-30T12:48:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2023-10-30T12:48:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2023-10-30T12:48:42.348 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2023-10-30T12:48:42.349 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2023-10-30T12:48:42.350 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2023-10-30T12:48:42.350 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2023-10-30T12:48:42.350 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2023-10-30T12:48:42.350 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2023-10-30T12:48:42.499 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2023-10-30T12:48:42.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2023-10-30T12:48:42.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2023-10-30T12:48:42.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2023-10-30T12:48:42.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2023-10-30T12:48:42.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2023-10-30T12:48:42.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2023-10-30T12:48:42.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2023-10-30T12:48:42.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2023-10-30T12:48:42.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2023-10-30T12:48:42.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2023-10-30T12:48:42.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2023-10-30T12:48:42.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2023-10-30T12:48:42.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2023-10-30T12:48:42.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2023-10-30T12:48:42.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2023-10-30T12:48:42.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2023-10-30T12:48:42.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2023-10-30T12:48:42.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2023-10-30T12:48:42.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2023-10-30T12:48:42.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2023-10-30T12:48:42.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2023-10-30T12:48:42.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2023-10-30T12:48:42.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2023-10-30T12:48:42.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2023-10-30T12:48:42.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2023-10-30T12:48:42.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2023-10-30T12:48:42.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2023-10-30T12:48:42.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2023-10-30T12:48:42.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2023-10-30T12:48:42.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2023-10-30T12:48:42.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2023-10-30T12:48:42.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2023-10-30T12:48:42.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2023-10-30T12:48:42.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2023-10-30T12:48:42.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2023-10-30T12:48:42.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2023-10-30T12:48:42.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2023-10-30T12:48:42.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2023-10-30T12:48:42.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2023-10-30T12:48:42.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2023-10-30T12:48:42.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2023-10-30T12:48:42.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2023-10-30T12:48:42.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2023-10-30T12:48:42.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2023-10-30T12:48:42.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2023-10-30T12:48:42.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2023-10-30T12:48:42.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2023-10-30T12:48:42.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2023-10-30T12:48:42.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2023-10-30T12:48:42.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2023-10-30T12:48:42.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2023-10-30T12:48:42.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2023-10-30T12:48:42.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2023-10-30T12:48:42.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2023-10-30T12:48:42.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2023-10-30T12:48:42.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2023-10-30T12:48:42.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2023-10-30T12:48:42.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2023-10-30T12:48:42.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2023-10-30T12:48:42.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2023-10-30T12:48:42.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2023-10-30T12:48:42.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2023-10-30T12:48:42.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2023-10-30T12:48:42.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2023-10-30T12:48:42.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2023-10-30T12:48:42.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2023-10-30T12:48:42.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2023-10-30T12:48:42.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2023-10-30T12:48:42.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2023-10-30T12:48:42.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2023-10-30T12:48:42.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2023-10-30T12:48:42.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2023-10-30T12:48:42.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2023-10-30T12:48:42.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2023-10-30T12:48:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2023-10-30T12:48:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2023-10-30T12:48:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2023-10-30T12:48:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2023-10-30T12:48:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2023-10-30T12:48:42.518 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2023-10-30T12:48:43.499 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2023-10-30T12:48:43.499 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2023-10-30T12:48:43.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2023-10-30T12:48:43.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2023-10-30T12:48:43.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2023-10-30T12:48:43.500 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2023-10-30T12:48:43.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2023-10-30T12:48:43.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2023-10-30T12:48:43.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2023-10-30T12:48:43.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2023-10-30T12:48:43.501 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2023-10-30T12:48:43.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2023-10-30T12:48:43.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2023-10-30T12:48:43.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2023-10-30T12:48:43.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2023-10-30T12:48:43.502 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2023-10-30T12:48:43.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2023-10-30T12:48:43.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2023-10-30T12:48:43.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2023-10-30T12:48:43.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2023-10-30T12:48:43.503 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2023-10-30T12:48:43.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2023-10-30T12:48:43.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2023-10-30T12:48:43.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2023-10-30T12:48:43.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2023-10-30T12:48:43.504 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2023-10-30T12:48:43.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2023-10-30T12:48:43.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2023-10-30T12:48:43.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2023-10-30T12:48:43.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2023-10-30T12:48:43.505 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2023-10-30T12:48:43.506 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2023-10-30T12:48:43.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2023-10-30T12:48:43.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2023-10-30T12:48:43.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2023-10-30T12:48:43.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2023-10-30T12:48:43.507 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2023-10-30T12:48:43.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2023-10-30T12:48:43.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2023-10-30T12:48:43.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2023-10-30T12:48:43.508 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2023-10-30T12:48:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2023-10-30T12:48:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2023-10-30T12:48:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2023-10-30T12:48:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2023-10-30T12:48:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2023-10-30T12:48:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2023-10-30T12:48:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2023-10-30T12:48:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2023-10-30T12:48:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2023-10-30T12:48:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2023-10-30T12:48:43.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2023-10-30T12:48:43.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2023-10-30T12:48:43.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2023-10-30T12:48:43.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2023-10-30T12:48:43.511 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2023-10-30T12:48:43.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2023-10-30T12:48:43.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2023-10-30T12:48:43.512 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2023-10-30T12:48:43.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2023-10-30T12:48:43.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2023-10-30T12:48:43.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2023-10-30T12:48:43.513 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2023-10-30T12:48:43.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2023-10-30T12:48:43.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2023-10-30T12:48:43.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2023-10-30T12:48:43.514 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2023-10-30T12:48:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2023-10-30T12:48:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2023-10-30T12:48:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2023-10-30T12:48:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2023-10-30T12:48:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2023-10-30T12:48:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2023-10-30T12:48:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2023-10-30T12:48:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2023-10-30T12:48:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2023-10-30T12:48:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2023-10-30T12:48:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2023-10-30T12:48:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2023-10-30T12:48:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2023-10-30T12:48:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2023-10-30T12:48:44.303 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:44.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:48:45.461 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2023-10-30T12:48:45.462 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2023-10-30T12:48:45.462 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2023-10-30T12:48:45.462 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2023-10-30T12:48:45.462 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2023-10-30T12:48:45.463 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2023-10-30T12:48:45.463 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2023-10-30T12:48:45.463 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2023-10-30T12:48:45.463 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2023-10-30T12:48:45.464 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2023-10-30T12:48:45.465 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2023-10-30T12:48:45.465 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2023-10-30T12:48:45.465 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2023-10-30T12:48:45.465 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2023-10-30T12:48:45.465 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2023-10-30T12:48:45.466 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2023-10-30T12:48:45.466 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2023-10-30T12:48:45.466 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2023-10-30T12:48:45.466 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2023-10-30T12:48:45.467 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2023-10-30T12:48:45.467 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2023-10-30T12:48:45.467 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2023-10-30T12:48:45.467 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2023-10-30T12:48:45.467 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2023-10-30T12:48:45.468 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2023-10-30T12:48:45.469 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2023-10-30T12:48:45.469 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2023-10-30T12:48:45.469 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2023-10-30T12:48:45.469 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2023-10-30T12:48:45.469 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2023-10-30T12:48:45.470 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2023-10-30T12:48:45.470 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2023-10-30T12:48:45.470 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2023-10-30T12:48:45.470 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2023-10-30T12:48:45.470 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2023-10-30T12:48:45.471 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2023-10-30T12:48:45.471 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2023-10-30T12:48:45.471 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2023-10-30T12:48:45.471 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2023-10-30T12:48:45.471 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2023-10-30T12:48:45.472 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2023-10-30T12:48:45.472 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2023-10-30T12:48:45.472 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2023-10-30T12:48:45.472 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2023-10-30T12:48:45.472 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2023-10-30T12:48:45.473 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2023-10-30T12:48:45.473 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2023-10-30T12:48:45.473 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2023-10-30T12:48:45.473 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2023-10-30T12:48:45.473 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2023-10-30T12:48:45.474 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2023-10-30T12:48:45.474 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2023-10-30T12:48:45.474 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2023-10-30T12:48:45.474 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2023-10-30T12:48:45.474 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2023-10-30T12:48:45.475 INFO:tasks.workunit.client.0.smithi008.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2023-10-30T12:48:45.475 INFO:tasks.workunit.client.0.smithi008.stdout:Take third snapshot .snap/test3 2023-10-30T12:48:45.480 INFO:tasks.workunit.client.0.smithi008.stdout:Delete the snapshots... 2023-10-30T12:48:46.786 INFO:tasks.workunit.client.0.smithi008.stdout:.//.snap/test1 2023-10-30T12:48:46.786 INFO:tasks.workunit.client.0.smithi008.stdout:.//.snap/test2 2023-10-30T12:48:46.786 INFO:tasks.workunit.client.0.smithi008.stdout:.//.snap/test3 2023-10-30T12:48:46.789 INFO:tasks.workunit.client.0.smithi008.stdout:./274/.snap/snap-subdir-test 2023-10-30T12:48:46.800 INFO:tasks.workunit.client.0.smithi008.stdout:./166/.snap/snap-subdir-test 2023-10-30T12:48:46.812 INFO:tasks.workunit.client.0.smithi008.stdout:./256/.snap/snap-subdir-test 2023-10-30T12:48:46.821 INFO:tasks.workunit.client.0.smithi008.stdout:./247/.snap/snap-subdir-test 2023-10-30T12:48:46.831 INFO:tasks.workunit.client.0.smithi008.stdout:./248/.snap/snap-subdir-test 2023-10-30T12:48:46.840 INFO:tasks.workunit.client.0.smithi008.stdout:./147/.snap/snap-subdir-test 2023-10-30T12:48:46.850 INFO:tasks.workunit.client.0.smithi008.stdout:./151/.snap/snap-subdir-test 2023-10-30T12:48:46.854 INFO:tasks.workunit.client.0.smithi008.stdout:./170/.snap/snap-subdir-test 2023-10-30T12:48:46.864 INFO:tasks.workunit.client.0.smithi008.stdout:./167/.snap/snap-subdir-test 2023-10-30T12:48:46.867 INFO:tasks.workunit.client.0.smithi008.stdout:./140/.snap/snap-subdir-test 2023-10-30T12:48:46.871 INFO:tasks.workunit.client.0.smithi008.stdout:./142/.snap/snap-subdir-test 2023-10-30T12:48:46.880 INFO:tasks.workunit.client.0.smithi008.stdout:./150/.snap/snap-subdir-test 2023-10-30T12:48:46.884 INFO:tasks.workunit.client.0.smithi008.stdout:./273/.snap/snap-subdir-test 2023-10-30T12:48:46.897 INFO:tasks.workunit.client.0.smithi008.stdout:./269/.snap/snap-subdir-test 2023-10-30T12:48:46.906 INFO:tasks.workunit.client.0.smithi008.stdout:./105/.snap/snap-subdir-test 2023-10-30T12:48:46.918 INFO:tasks.workunit.client.0.smithi008.stdout:./210/.snap/snap-subdir-test 2023-10-30T12:48:46.921 INFO:tasks.workunit.client.0.smithi008.stdout:./265/.snap/snap-subdir-test 2023-10-30T12:48:46.933 INFO:tasks.workunit.client.0.smithi008.stdout:./218/.snap/snap-subdir-test 2023-10-30T12:48:46.944 INFO:tasks.workunit.client.0.smithi008.stdout:./290/.snap/snap-subdir-test 2023-10-30T12:48:47.769 INFO:tasks.workunit.client.0.smithi008.stdout:./149/.snap/snap-subdir-test 2023-10-30T12:48:47.776 INFO:tasks.workunit.client.0.smithi008.stdout:./109/.snap/snap-subdir-test 2023-10-30T12:48:47.789 INFO:tasks.workunit.client.0.smithi008.stdout:./292/.snap/snap-subdir-test 2023-10-30T12:48:47.793 INFO:tasks.workunit.client.0.smithi008.stdout:./242/.snap/snap-subdir-test 2023-10-30T12:48:47.804 INFO:tasks.workunit.client.0.smithi008.stdout:./158/.snap/snap-subdir-test 2023-10-30T12:48:47.807 INFO:tasks.workunit.client.0.smithi008.stdout:./243/.snap/snap-subdir-test 2023-10-30T12:48:47.817 INFO:tasks.workunit.client.0.smithi008.stdout:./106/.snap/snap-subdir-test 2023-10-30T12:48:47.831 INFO:tasks.workunit.client.0.smithi008.stdout:./250/.snap/snap-subdir-test 2023-10-30T12:48:47.842 INFO:tasks.workunit.client.0.smithi008.stdout:./221/.snap/snap-subdir-test 2023-10-30T12:48:47.856 INFO:tasks.workunit.client.0.smithi008.stdout:./244/.snap/snap-subdir-test 2023-10-30T12:48:47.869 INFO:tasks.workunit.client.0.smithi008.stdout:./258/.snap/snap-subdir-test 2023-10-30T12:48:47.882 INFO:tasks.workunit.client.0.smithi008.stdout:./268/.snap/snap-subdir-test 2023-10-30T12:48:47.886 INFO:tasks.workunit.client.0.smithi008.stdout:./189/.snap/snap-subdir-test 2023-10-30T12:48:47.897 INFO:tasks.workunit.client.0.smithi008.stdout:./285/.snap/snap-subdir-test 2023-10-30T12:48:47.910 INFO:tasks.workunit.client.0.smithi008.stdout:./208/.snap/snap-subdir-test 2023-10-30T12:48:47.921 INFO:tasks.workunit.client.0.smithi008.stdout:./288/.snap/snap-subdir-test 2023-10-30T12:48:47.932 INFO:tasks.workunit.client.0.smithi008.stdout:./209/.snap/snap-subdir-test 2023-10-30T12:48:47.946 INFO:tasks.workunit.client.0.smithi008.stdout:./253/.snap/snap-subdir-test 2023-10-30T12:48:48.388 INFO:tasks.workunit.client.0.smithi008.stdout:./249/.snap/snap-subdir-test 2023-10-30T12:48:48.399 INFO:tasks.workunit.client.0.smithi008.stdout:./133/.snap/snap-subdir-test 2023-10-30T12:48:48.410 INFO:tasks.workunit.client.0.smithi008.stdout:./291/.snap/snap-subdir-test 2023-10-30T12:48:49.078 INFO:tasks.workunit.client.0.smithi008.stdout:./226/.snap/snap-subdir-test 2023-10-30T12:48:49.090 INFO:tasks.workunit.client.0.smithi008.stdout:./173/.snap/snap-subdir-test 2023-10-30T12:48:49.102 INFO:tasks.workunit.client.0.smithi008.stdout:./174/.snap/snap-subdir-test 2023-10-30T12:48:49.113 INFO:tasks.workunit.client.0.smithi008.stdout:./299/.snap/snap-subdir-test 2023-10-30T12:48:49.201 INFO:tasks.workunit.client.0.smithi008.stdout:./262/.snap/snap-subdir-test 2023-10-30T12:48:49.772 INFO:tasks.workunit.client.0.smithi008.stdout:./231/.snap/snap-subdir-test 2023-10-30T12:48:49.786 INFO:tasks.workunit.client.0.smithi008.stdout:./214/.snap/snap-subdir-test 2023-10-30T12:48:49.800 INFO:tasks.workunit.client.0.smithi008.stdout:./281/.snap/snap-subdir-test 2023-10-30T12:48:49.812 INFO:tasks.workunit.client.0.smithi008.stdout:./289/.snap/snap-subdir-test 2023-10-30T12:48:49.828 INFO:tasks.workunit.client.0.smithi008.stdout:./283/.snap/snap-subdir-test 2023-10-30T12:48:49.842 INFO:tasks.workunit.client.0.smithi008.stdout:./232/.snap/snap-subdir-test 2023-10-30T12:48:49.852 INFO:tasks.workunit.client.0.smithi008.stdout:./298/.snap/snap-subdir-test 2023-10-30T12:48:50.720 INFO:tasks.workunit.client.0.smithi008.stdout:./233/.snap/snap-subdir-test 2023-10-30T12:48:50.729 INFO:tasks.workunit.client.0.smithi008.stdout:./155/.snap/snap-subdir-test 2023-10-30T12:48:50.741 INFO:tasks.workunit.client.0.smithi008.stdout:./114/.snap/snap-subdir-test 2023-10-30T12:48:50.748 INFO:tasks.workunit.client.0.smithi008.stdout:./159/.snap/snap-subdir-test 2023-10-30T12:48:51.140 INFO:tasks.workunit.client.0.smithi008.stdout:./187/.snap/snap-subdir-test 2023-10-30T12:48:51.152 INFO:tasks.workunit.client.0.smithi008.stdout:./185/.snap/snap-subdir-test 2023-10-30T12:48:51.164 INFO:tasks.workunit.client.0.smithi008.stdout:./293/.snap/snap-subdir-test 2023-10-30T12:48:51.684 INFO:tasks.workunit.client.0.smithi008.stdout:./228/.snap/snap-subdir-test 2023-10-30T12:48:51.693 INFO:tasks.workunit.client.0.smithi008.stdout:./120/.snap/snap-subdir-test 2023-10-30T12:48:51.705 INFO:tasks.workunit.client.0.smithi008.stdout:./110/.snap/snap-subdir-test 2023-10-30T12:48:51.718 INFO:tasks.workunit.client.0.smithi008.stdout:./267/.snap/snap-subdir-test 2023-10-30T12:48:51.729 INFO:tasks.workunit.client.0.smithi008.stdout:./202/.snap/snap-subdir-test 2023-10-30T12:48:51.740 INFO:tasks.workunit.client.0.smithi008.stdout:./137/.snap/snap-subdir-test 2023-10-30T12:48:51.753 INFO:tasks.workunit.client.0.smithi008.stdout:./246/.snap/snap-subdir-test 2023-10-30T12:48:51.764 INFO:tasks.workunit.client.0.smithi008.stdout:./219/.snap/snap-subdir-test 2023-10-30T12:48:51.776 INFO:tasks.workunit.client.0.smithi008.stdout:./198/.snap/snap-subdir-test 2023-10-30T12:48:51.787 INFO:tasks.workunit.client.0.smithi008.stdout:./136/.snap/snap-subdir-test 2023-10-30T12:48:51.800 INFO:tasks.workunit.client.0.smithi008.stdout:./270/.snap/snap-subdir-test 2023-10-30T12:48:51.810 INFO:tasks.workunit.client.0.smithi008.stdout:./125/.snap/snap-subdir-test 2023-10-30T12:48:51.824 INFO:tasks.workunit.client.0.smithi008.stdout:./194/.snap/snap-subdir-test 2023-10-30T12:48:51.834 INFO:tasks.workunit.client.0.smithi008.stdout:./119/.snap/snap-subdir-test 2023-10-30T12:48:51.844 INFO:tasks.workunit.client.0.smithi008.stdout:./297/.snap/snap-subdir-test 2023-10-30T12:48:51.856 INFO:tasks.workunit.client.0.smithi008.stdout:./230/.snap/snap-subdir-test 2023-10-30T12:48:51.869 INFO:tasks.workunit.client.0.smithi008.stdout:./260/.snap/snap-subdir-test 2023-10-30T12:48:51.878 INFO:tasks.workunit.client.0.smithi008.stdout:./157/.snap/snap-subdir-test 2023-10-30T12:48:51.889 INFO:tasks.workunit.client.0.smithi008.stdout:./172/.snap/snap-subdir-test 2023-10-30T12:48:51.900 INFO:tasks.workunit.client.0.smithi008.stdout:./277/.snap/snap-subdir-test 2023-10-30T12:48:51.913 INFO:tasks.workunit.client.0.smithi008.stdout:./186/.snap/snap-subdir-test 2023-10-30T12:48:51.924 INFO:tasks.workunit.client.0.smithi008.stdout:./237/.snap/snap-subdir-test 2023-10-30T12:48:52.186 INFO:tasks.workunit.client.0.smithi008.stdout:./190/.snap/snap-subdir-test 2023-10-30T12:48:52.200 INFO:tasks.workunit.client.0.smithi008.stdout:./213/.snap/snap-subdir-test 2023-10-30T12:48:52.210 INFO:tasks.workunit.client.0.smithi008.stdout:./104/.snap/snap-subdir-test 2023-10-30T12:48:52.219 INFO:tasks.workunit.client.0.smithi008.stdout:./164/.snap/snap-subdir-test 2023-10-30T12:48:52.228 INFO:tasks.workunit.client.0.smithi008.stdout:./112/.snap/snap-subdir-test 2023-10-30T12:48:52.240 INFO:tasks.workunit.client.0.smithi008.stdout:./295/.snap/snap-subdir-test 2023-10-30T12:48:52.505 INFO:tasks.workunit.client.0.smithi008.stdout:./272/.snap/snap-subdir-test 2023-10-30T12:48:52.512 INFO:tasks.workunit.client.0.smithi008.stdout:./100/.snap/snap-subdir-test 2023-10-30T12:48:52.524 INFO:tasks.workunit.client.0.smithi008.stdout:./134/.snap/snap-subdir-test 2023-10-30T12:48:52.535 INFO:tasks.workunit.client.0.smithi008.stdout:./200/.snap/snap-subdir-test 2023-10-30T12:48:52.547 INFO:tasks.workunit.client.0.smithi008.stdout:./115/.snap/snap-subdir-test 2023-10-30T12:48:52.559 INFO:tasks.workunit.client.0.smithi008.stdout:./284/.snap/snap-subdir-test 2023-10-30T12:48:52.572 INFO:tasks.workunit.client.0.smithi008.stdout:./280/.snap/snap-subdir-test 2023-10-30T12:48:52.581 INFO:tasks.workunit.client.0.smithi008.stdout:./116/.snap/snap-subdir-test 2023-10-30T12:48:52.589 INFO:tasks.workunit.client.0.smithi008.stdout:./245/.snap/snap-subdir-test 2023-10-30T12:48:52.601 INFO:tasks.workunit.client.0.smithi008.stdout:./195/.snap/snap-subdir-test 2023-10-30T12:48:52.963 INFO:tasks.workunit.client.0.smithi008.stdout:./138/.snap/snap-subdir-test 2023-10-30T12:48:52.972 INFO:tasks.workunit.client.0.smithi008.stdout:./135/.snap/snap-subdir-test 2023-10-30T12:48:52.980 INFO:tasks.workunit.client.0.smithi008.stdout:./117/.snap/snap-subdir-test 2023-10-30T12:48:52.991 INFO:tasks.workunit.client.0.smithi008.stdout:./241/.snap/snap-subdir-test 2023-10-30T12:48:53.000 INFO:tasks.workunit.client.0.smithi008.stdout:./143/.snap/snap-subdir-test 2023-10-30T12:48:53.009 INFO:tasks.workunit.client.0.smithi008.stdout:./180/.snap/snap-subdir-test 2023-10-30T12:48:53.017 INFO:tasks.workunit.client.0.smithi008.stdout:./126/.snap/snap-subdir-test 2023-10-30T12:48:53.024 INFO:tasks.workunit.client.0.smithi008.stdout:./161/.snap/snap-subdir-test 2023-10-30T12:48:53.030 INFO:tasks.workunit.client.0.smithi008.stdout:./204/.snap/snap-subdir-test 2023-10-30T12:48:53.040 INFO:tasks.workunit.client.0.smithi008.stdout:./176/.snap/snap-subdir-test 2023-10-30T12:48:53.049 INFO:tasks.workunit.client.0.smithi008.stdout:./255/.snap/snap-subdir-test 2023-10-30T12:48:53.058 INFO:tasks.workunit.client.0.smithi008.stdout:./215/.snap/snap-subdir-test 2023-10-30T12:48:53.065 INFO:tasks.workunit.client.0.smithi008.stdout:./111/.snap/snap-subdir-test 2023-10-30T12:48:53.074 INFO:tasks.workunit.client.0.smithi008.stdout:./238/.snap/snap-subdir-test 2023-10-30T12:48:53.077 INFO:tasks.workunit.client.0.smithi008.stdout:./191/.snap/snap-subdir-test 2023-10-30T12:48:53.085 INFO:tasks.workunit.client.0.smithi008.stdout:./184/.snap/snap-subdir-test 2023-10-30T12:48:53.088 INFO:tasks.workunit.client.0.smithi008.stdout:./192/.snap/snap-subdir-test 2023-10-30T12:48:53.095 INFO:tasks.workunit.client.0.smithi008.stdout:./103/.snap/snap-subdir-test 2023-10-30T12:48:53.101 INFO:tasks.workunit.client.0.smithi008.stdout:./128/.snap/snap-subdir-test 2023-10-30T12:48:53.108 INFO:tasks.workunit.client.0.smithi008.stdout:./229/.snap/snap-subdir-test 2023-10-30T12:48:53.932 INFO:tasks.workunit.client.0.smithi008.stdout:./144/.snap/snap-subdir-test 2023-10-30T12:48:54.151 INFO:tasks.workunit.client.0.smithi008.stdout:./146/.snap/snap-subdir-test 2023-10-30T12:48:54.613 INFO:tasks.workunit.client.0.smithi008.stdout:./227/.snap/snap-subdir-test 2023-10-30T12:48:54.622 INFO:tasks.workunit.client.0.smithi008.stdout:./278/.snap/snap-subdir-test 2023-10-30T12:48:54.632 INFO:tasks.workunit.client.0.smithi008.stdout:./141/.snap/snap-subdir-test 2023-10-30T12:48:54.641 INFO:tasks.workunit.client.0.smithi008.stdout:./234/.snap/snap-subdir-test 2023-10-30T12:48:54.650 INFO:tasks.workunit.client.0.smithi008.stdout:./225/.snap/snap-subdir-test 2023-10-30T12:48:54.666 INFO:tasks.workunit.client.0.smithi008.stdout:./182/.snap/snap-subdir-test 2023-10-30T12:48:54.674 INFO:tasks.workunit.client.0.smithi008.stdout:./107/.snap/snap-subdir-test 2023-10-30T12:48:54.682 INFO:tasks.workunit.client.0.smithi008.stdout:./178/.snap/snap-subdir-test 2023-10-30T12:48:54.689 INFO:tasks.workunit.client.0.smithi008.stdout:./266/.snap/snap-subdir-test 2023-10-30T12:48:54.695 INFO:tasks.workunit.client.0.smithi008.stdout:./240/.snap/snap-subdir-test 2023-10-30T12:48:54.704 INFO:tasks.workunit.client.0.smithi008.stdout:./156/.snap/snap-subdir-test 2023-10-30T12:48:56.124 INFO:tasks.workunit.client.0.smithi008.stdout:./131/.snap/snap-subdir-test 2023-10-30T12:48:56.132 INFO:tasks.workunit.client.0.smithi008.stdout:./123/.snap/snap-subdir-test 2023-10-30T12:48:56.139 INFO:tasks.workunit.client.0.smithi008.stdout:./160/.snap/snap-subdir-test 2023-10-30T12:48:56.146 INFO:tasks.workunit.client.0.smithi008.stdout:./165/.snap/snap-subdir-test 2023-10-30T12:48:56.246 INFO:tasks.workunit.client.0.smithi008.stdout:./101/.snap/snap-subdir-test 2023-10-30T12:48:56.253 INFO:tasks.workunit.client.0.smithi008.stdout:./148/.snap/snap-subdir-test 2023-10-30T12:48:56.581 INFO:tasks.workunit.client.0.smithi008.stdout:./168/.snap/snap-subdir-test 2023-10-30T12:48:56.590 INFO:tasks.workunit.client.0.smithi008.stdout:./179/.snap/snap-subdir-test 2023-10-30T12:48:56.601 INFO:tasks.workunit.client.0.smithi008.stdout:./257/.snap/snap-subdir-test 2023-10-30T12:48:56.609 INFO:tasks.workunit.client.0.smithi008.stdout:./252/.snap/snap-subdir-test 2023-10-30T12:48:56.619 INFO:tasks.workunit.client.0.smithi008.stdout:./132/.snap/snap-subdir-test 2023-10-30T12:48:56.627 INFO:tasks.workunit.client.0.smithi008.stdout:./236/.snap/snap-subdir-test 2023-10-30T12:48:57.098 INFO:tasks.workunit.client.0.smithi008.stdout:./211/.snap/snap-subdir-test 2023-10-30T12:48:57.101 INFO:tasks.workunit.client.0.smithi008.stdout:./199/.snap/snap-subdir-test 2023-10-30T12:48:57.112 INFO:tasks.workunit.client.0.smithi008.stdout:./205/.snap/snap-subdir-test 2023-10-30T12:48:57.120 INFO:tasks.workunit.client.0.smithi008.stdout:./163/.snap/snap-subdir-test 2023-10-30T12:48:57.129 INFO:tasks.workunit.client.0.smithi008.stdout:./261/.snap/snap-subdir-test 2023-10-30T12:48:58.092 INFO:tasks.workunit.client.0.smithi008.stdout:./201/.snap/snap-subdir-test 2023-10-30T12:48:58.100 INFO:tasks.workunit.client.0.smithi008.stdout:./282/.snap/snap-subdir-test 2023-10-30T12:48:58.109 INFO:tasks.workunit.client.0.smithi008.stdout:./296/.snap/snap-subdir-test 2023-10-30T12:48:58.116 INFO:tasks.workunit.client.0.smithi008.stdout:./217/.snap/snap-subdir-test 2023-10-30T12:48:58.124 INFO:tasks.workunit.client.0.smithi008.stdout:./153/.snap/snap-subdir-test 2023-10-30T12:48:58.131 INFO:tasks.workunit.client.0.smithi008.stdout:./206/.snap/snap-subdir-test 2023-10-30T12:48:58.136 INFO:tasks.workunit.client.0.smithi008.stdout:./122/.snap/snap-subdir-test 2023-10-30T12:48:58.142 INFO:tasks.workunit.client.0.smithi008.stdout:./162/.snap/snap-subdir-test 2023-10-30T12:48:58.145 INFO:tasks.workunit.client.0.smithi008.stdout:./197/.snap/snap-subdir-test 2023-10-30T12:48:58.153 INFO:tasks.workunit.client.0.smithi008.stdout:./216/.snap/snap-subdir-test 2023-10-30T12:48:59.088 INFO:tasks.workunit.client.0.smithi008.stdout:./254/.snap/snap-subdir-test 2023-10-30T12:48:59.094 INFO:tasks.workunit.client.0.smithi008.stdout:./113/.snap/snap-subdir-test 2023-10-30T12:48:59.100 INFO:tasks.workunit.client.0.smithi008.stdout:./152/.snap/snap-subdir-test 2023-10-30T12:48:59.107 INFO:tasks.workunit.client.0.smithi008.stdout:./121/.snap/snap-subdir-test 2023-10-30T12:48:59.114 INFO:tasks.workunit.client.0.smithi008.stdout:./118/.snap/snap-subdir-test 2023-10-30T12:48:59.117 INFO:tasks.workunit.client.0.smithi008.stdout:./196/.snap/snap-subdir-test 2023-10-30T12:48:59.125 INFO:tasks.workunit.client.0.smithi008.stdout:./124/.snap/snap-subdir-test 2023-10-30T12:48:59.134 INFO:tasks.workunit.client.0.smithi008.stdout:./239/.snap/snap-subdir-test 2023-10-30T12:48:59.137 INFO:tasks.workunit.client.0.smithi008.stdout:./188/.snap/snap-subdir-test 2023-10-30T12:48:59.146 INFO:tasks.workunit.client.0.smithi008.stdout:./287/.snap/snap-subdir-test 2023-10-30T12:48:59.157 INFO:tasks.workunit.client.0.smithi008.stdout:./275/.snap/snap-subdir-test 2023-10-30T12:48:59.166 INFO:tasks.workunit.client.0.smithi008.stdout:./259/.snap/snap-subdir-test 2023-10-30T12:48:59.174 INFO:tasks.workunit.client.0.smithi008.stdout:./224/.snap/snap-subdir-test 2023-10-30T12:48:59.508 INFO:tasks.workunit.client.0.smithi008.stdout:./264/.snap/snap-subdir-test 2023-10-30T12:48:59.516 INFO:tasks.workunit.client.0.smithi008.stdout:./145/.snap/snap-subdir-test 2023-10-30T12:48:59.525 INFO:tasks.workunit.client.0.smithi008.stdout:./279/.snap/snap-subdir-test 2023-10-30T12:48:59.538 INFO:tasks.workunit.client.0.smithi008.stdout:./263/.snap/snap-subdir-test 2023-10-30T12:48:59.548 INFO:tasks.workunit.client.0.smithi008.stdout:./130/.snap/snap-subdir-test 2023-10-30T12:48:59.561 INFO:tasks.workunit.client.0.smithi008.stdout:./193/.snap/snap-subdir-test 2023-10-30T12:49:00.534 INFO:tasks.workunit.client.0.smithi008.stdout:./223/.snap/snap-subdir-test 2023-10-30T12:49:00.548 INFO:tasks.workunit.client.0.smithi008.stdout:./207/.snap/snap-subdir-test 2023-10-30T12:49:00.561 INFO:tasks.workunit.client.0.smithi008.stdout:./276/.snap/snap-subdir-test 2023-10-30T12:49:00.574 INFO:tasks.workunit.client.0.smithi008.stdout:./271/.snap/snap-subdir-test 2023-10-30T12:49:00.586 INFO:tasks.workunit.client.0.smithi008.stdout:./251/.snap/snap-subdir-test 2023-10-30T12:49:00.848 INFO:tasks.workunit.client.0.smithi008.stdout:./175/.snap/snap-subdir-test 2023-10-30T12:49:00.860 INFO:tasks.workunit.client.0.smithi008.stdout:./222/.snap/snap-subdir-test 2023-10-30T12:49:00.873 INFO:tasks.workunit.client.0.smithi008.stdout:./220/.snap/snap-subdir-test 2023-10-30T12:49:01.077 INFO:tasks.workunit.client.0.smithi008.stdout:./203/.snap/snap-subdir-test 2023-10-30T12:49:01.606 INFO:tasks.workunit.client.0.smithi008.stdout:./181/.snap/snap-subdir-test 2023-10-30T12:49:01.618 INFO:tasks.workunit.client.0.smithi008.stdout:./286/.snap/snap-subdir-test 2023-10-30T12:49:01.630 INFO:tasks.workunit.client.0.smithi008.stdout:./127/.snap/snap-subdir-test 2023-10-30T12:49:01.641 INFO:tasks.workunit.client.0.smithi008.stdout:./108/.snap/snap-subdir-test 2023-10-30T12:49:01.653 INFO:tasks.workunit.client.0.smithi008.stdout:./183/.snap/snap-subdir-test 2023-10-30T12:49:01.665 INFO:tasks.workunit.client.0.smithi008.stdout:./294/.snap/snap-subdir-test 2023-10-30T12:49:01.677 INFO:tasks.workunit.client.0.smithi008.stdout:./235/.snap/snap-subdir-test 2023-10-30T12:49:01.697 INFO:tasks.workunit.client.0.smithi008.stdout:./177/.snap/snap-subdir-test 2023-10-30T12:49:01.709 INFO:tasks.workunit.client.0.smithi008.stdout:./169/.snap/snap-subdir-test 2023-10-30T12:49:01.721 INFO:tasks.workunit.client.0.smithi008.stdout:./212/.snap/snap-subdir-test 2023-10-30T12:49:01.730 INFO:tasks.workunit.client.0.smithi008.stdout:./154/.snap/snap-subdir-test 2023-10-30T12:49:01.741 INFO:tasks.workunit.client.0.smithi008.stdout:./171/.snap/snap-subdir-test 2023-10-30T12:49:01.951 INFO:tasks.workunit.client.0.smithi008.stdout:./139/.snap/snap-subdir-test 2023-10-30T12:49:01.962 INFO:tasks.workunit.client.0.smithi008.stdout:./102/.snap/snap-subdir-test 2023-10-30T12:49:01.972 INFO:tasks.workunit.client.0.smithi008.stdout:./129/.snap/snap-subdir-test 2023-10-30T12:49:11.959 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:12.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.110+0000 7ff237a54640 1 Processor -- start 2023-10-30T12:49:12.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.110+0000 7ff237a54640 1 -- start start 2023-10-30T12:49:12.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff237a54640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff230151840 0x7ff2300c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff237a54640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff237a54640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2300c5030 con 0x7ff2300cb8b0 2023-10-30T12:49:12.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff237a54640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff230157830 con 0x7ff230151840 2023-10-30T12:49:12.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff237a54640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff230157b10 con 0x7ff230151c20 2023-10-30T12:49:12.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff230151840 0x7ff2300c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff230151840 0x7ff2300c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56244/0 (socket says 172.21.15.8:56244) 2023-10-30T12:49:12.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 -- 172.21.15.8:0/2545094413 learned_addr learned my addr 172.21.15.8:0/2545094413 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:12.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff234fef640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 -- 172.21.15.8:0/2545094413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 msgr2=0x7ff230072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 -- 172.21.15.8:0/2545094413 >> v1:172.21.15.46:6789/0 conn(0x7ff2300cb8b0 legacy=0x7ff23006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.114+0000 7ff2357f0640 1 -- 172.21.15.8:0/2545094413 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230157df0 con 0x7ff230151840 2023-10-30T12:49:12.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff2357f0640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff230151840 0x7ff2300c49a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009870 tx=0x7ff22c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e2d820e02bc0f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff227fff640 1 -- 172.21.15.8:0/2545094413 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0a8a30 con 0x7ff230151840 2023-10-30T12:49:12.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff227fff640 1 -- 172.21.15.8:0/2545094413 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c0a8b90 con 0x7ff230151840 2023-10-30T12:49:12.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff227fff640 1 -- 172.21.15.8:0/2545094413 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0b0440 con 0x7ff230151840 2023-10-30T12:49:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- 172.21.15.8:0/2545094413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff230151840 msgr2=0x7ff2300c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff230151840 0x7ff2300c49a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009870 tx=0x7ff22c01ed30 comp rx=0 tx=0).stop 2023-10-30T12:49:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- 172.21.15.8:0/2545094413 shutdown_connections 2023-10-30T12:49:12.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 --2- 172.21.15.8:0/2545094413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff230151840 0x7ff2300c49a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- 172.21.15.8:0/2545094413 >> 172.21.15.8:0/2545094413 conn(0x7ff2300cac70 msgr2=0x7ff2300cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- 172.21.15.8:0/2545094413 shutdown_connections 2023-10-30T12:49:12.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- 172.21.15.8:0/2545094413 wait complete. 2023-10-30T12:49:12.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 Processor -- start 2023-10-30T12:49:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.118+0000 7ff237a54640 1 -- start start 2023-10-30T12:49:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff230151840 0x7ff2301466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff230071410 con 0x7ff2300cb8b0 2023-10-30T12:49:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff230070a40 con 0x7ff230151840 2023-10-30T12:49:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301ef1f0 con 0x7ff230151c20 2023-10-30T12:49:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56256/0 (socket says 172.21.15.8:56256) 2023-10-30T12:49:12.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 -- 172.21.15.8:0/3093307010 learned_addr learned my addr 172.21.15.8:0/3093307010 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:12.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff234fef640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff230151840 0x7ff2301466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff235ff1640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 -- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 msgr2=0x7ff230146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 -- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff230151840 msgr2=0x7ff2301466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff230151840 0x7ff2301466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 -- 172.21.15.8:0/3093307010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c0094d0 con 0x7ff2300cb8b0 2023-10-30T12:49:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff2357f0640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0b3040 tx=0x7ff22c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0ba600 con 0x7ff2300cb8b0 2023-10-30T12:49:12.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c0ba780 con 0x7ff2300cb8b0 2023-10-30T12:49:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0bab30 con 0x7ff2300cb8b0 2023-10-30T12:49:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2301ef730 con 0x7ff2300cb8b0 2023-10-30T12:49:12.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff230070110 con 0x7ff2300cb8b0 2023-10-30T12:49:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23004e170 con 0x7ff2300cb8b0 2023-10-30T12:49:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.122+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff22c0bac90 con 0x7ff2300cb8b0 2023-10-30T12:49:12.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.126+0000 7ff225ffb640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 0x7ff214068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.126+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 7564+0+0 (secure 0 0 0) 0x7ff22c122c00 con 0x7ff2300cb8b0 2023-10-30T12:49:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.126+0000 7ff234fef640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 0x7ff214068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.126+0000 7ff234fef640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 0x7ff214068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff230148050 tx=0x7ff220008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.130+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff22c0ef9c0 con 0x7ff2300cb8b0 2023-10-30T12:49:12.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff230066510 con 0x7ff2300cb8b0 2023-10-30T12:49:12.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff225ffb640 1 -- 172.21.15.8:0/3093307010 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7ff22c0b5020 con 0x7ff2300cb8b0 2023-10-30T12:49:12.328 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:12.328 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 msgr2=0x7ff214068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 0x7ff214068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff230148050 tx=0x7ff220008040 comp rx=0 tx=0).stop 2023-10-30T12:49:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 msgr2=0x7ff230070ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0b3040 tx=0x7ff22c01bad0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 shutdown_connections 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2140664a0 0x7ff214068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff230151840 0x7ff2301466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff230151c20 0x7ff230146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 --2- 172.21.15.8:0/3093307010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2300cb8b0 0x7ff230070ed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 >> 172.21.15.8:0/3093307010 conn(0x7ff2300cac70 msgr2=0x7ff230072000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 shutdown_connections 2023-10-30T12:49:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.322+0000 7ff237a54640 1 -- 172.21.15.8:0/3093307010 wait complete. 2023-10-30T12:49:12.348 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2023-10-30T12:49:12.349 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:49:12.349 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:49:12.364 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:49:12.364 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:12.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.542+0000 7f0e3d371640 1 Processor -- start 2023-10-30T12:49:12.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.542+0000 7f0e3d371640 1 -- start start 2023-10-30T12:49:12.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3d371640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 0x7f0e3815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3d371640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0e3806aeb0 0x7f0e38069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3d371640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3815dcc0 con 0x7f0e380c8eb0 2023-10-30T12:49:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3d371640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e38159800 con 0x7f0e3806aeb0 2023-10-30T12:49:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3d371640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38159d80 con 0x7f0e3815ac70 2023-10-30T12:49:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0e3806aeb0 0x7f0e38069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e37fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0e380c8eb0 0x7f0e3815a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60054/0 (socket says 172.21.15.8:60054) 2023-10-30T12:49:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0e3806aeb0 0x7f0e38069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56258/0 (socket says 172.21.15.8:56258) 2023-10-30T12:49:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e37fff640 1 -- 172.21.15.8:0/3321701121 learned_addr learned my addr 172.21.15.8:0/3321701121 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e377fe640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 0x7f0e3815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 -- 172.21.15.8:0/3321701121 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 msgr2=0x7f0e3815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 0x7f0e3815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 -- 172.21.15.8:0/3321701121 >> v1:172.21.15.8:6789/0 conn(0x7f0e380c8eb0 legacy=0x7f0e3815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e3cb70640 1 -- 172.21.15.8:0/3321701121 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3815d420 con 0x7f0e3806aeb0 2023-10-30T12:49:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e36ffd640 1 -- 172.21.15.8:0/3321701121 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923889715 0 0) 0x7f0e28001120 con 0x7f0e380c8eb0 2023-10-30T12:49:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.546+0000 7f0e377fe640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 0x7f0e3815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:12.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3cb70640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38069e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009d90 tx=0x7f0e3001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8b96eb0497a4ad server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e36ffd640 1 -- 172.21.15.8:0/3321701121 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300af600 con 0x7f0e3806aeb0 2023-10-30T12:49:12.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e36ffd640 1 -- 172.21.15.8:0/3321701121 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300ab050 con 0x7f0e3806aeb0 2023-10-30T12:49:12.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e36ffd640 1 -- 172.21.15.8:0/3321701121 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300af980 con 0x7f0e3806aeb0 2023-10-30T12:49:12.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- 172.21.15.8:0/3321701121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 msgr2=0x7f0e38069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38069e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009d90 tx=0x7f0e3001f130 comp rx=0 tx=0).stop 2023-10-30T12:49:12.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- 172.21.15.8:0/3321701121 shutdown_connections 2023-10-30T12:49:12.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38069e90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 --2- 172.21.15.8:0/3321701121 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e3815ac70 0x7f0e3815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- 172.21.15.8:0/3321701121 >> 172.21.15.8:0/3321701121 conn(0x7f0e380cef00 msgr2=0x7f0e380cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:12.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- 172.21.15.8:0/3321701121 shutdown_connections 2023-10-30T12:49:12.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- 172.21.15.8:0/3321701121 wait complete. 2023-10-30T12:49:12.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 Processor -- start 2023-10-30T12:49:12.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.550+0000 7f0e3d371640 1 -- start start 2023-10-30T12:49:12.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e380c8eb0 0x7f0e381e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3815ac70 0x7f0e380c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380c3c30 con 0x7f0e3806aeb0 2023-10-30T12:49:12.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38158580 con 0x7f0e3815ac70 2023-10-30T12:49:12.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38158ab0 con 0x7f0e380c8eb0 2023-10-30T12:49:12.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56272/0 (socket says 172.21.15.8:56272) 2023-10-30T12:49:12.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 -- 172.21.15.8:0/456887523 learned_addr learned my addr 172.21.15.8:0/456887523 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:12.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e377fe640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e380c8eb0 0x7f0e381e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3cb70640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3815ac70 0x7f0e380c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 -- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e380c8eb0 msgr2=0x7f0e381e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e380c8eb0 0x7f0e381e6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 -- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3815ac70 msgr2=0x7f0e380c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3815ac70 0x7f0e380c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 -- 172.21.15.8:0/456887523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30009a70 con 0x7f0e3806aeb0 2023-10-30T12:49:12.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e37fff640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7b0 tx=0x7f0e2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280072d0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28007450 con 0x7f0e3806aeb0 2023-10-30T12:49:12.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e38152730 con 0x7f0e3806aeb0 2023-10-30T12:49:12.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280047f0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e38153600 con 0x7f0e3806aeb0 2023-10-30T12:49:12.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3804e170 con 0x7f0e3806aeb0 2023-10-30T12:49:12.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.554+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0e28004950 con 0x7f0e3806aeb0 2023-10-30T12:49:12.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.558+0000 7f0e34ff9640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 0x7f0e24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:12.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.558+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 7564+0+0 (secure 0 0 0) 0x7f0e280838e0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.558+0000 7f0e377fe640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 0x7f0e24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:12.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.558+0000 7f0e377fe640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 0x7f0e24068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e380c7060 tx=0x7f0e2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:12.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.562+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0e280506b0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.750+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e3815dcc0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.750+0000 7f0e34ff9640 1 -- 172.21.15.8:0/456887523 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f0e3815dcc0 con 0x7f0e3806aeb0 2023-10-30T12:49:12.757 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:12.758 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:12.758 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:12.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 msgr2=0x7f0e24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 0x7f0e24068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e380c7060 tx=0x7f0e2c008040 comp rx=0 tx=0).stop 2023-10-30T12:49:12.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 msgr2=0x7f0e38152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:12.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7b0 tx=0x7f0e2800cc70 comp rx=0 tx=0).stop 2023-10-30T12:49:12.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 shutdown_connections 2023-10-30T12:49:12.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0e240664a0 0x7f0e24068960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3815ac70 0x7f0e380c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0e380c8eb0 0x7f0e381e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 --2- 172.21.15.8:0/456887523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e3806aeb0 0x7f0e38152b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:12.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 >> 172.21.15.8:0/456887523 conn(0x7f0e380cef00 msgr2=0x7f0e3815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:12.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 shutdown_connections 2023-10-30T12:49:12.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:12.754+0000 7f0e3d371640 1 -- 172.21.15.8:0/456887523 wait complete. 2023-10-30T12:49:12.778 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:14.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:14.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:14.779 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:14.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 Processor -- start 2023-10-30T12:49:14.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 -- start start 2023-10-30T12:49:14.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e24151840 0x7f9e240c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e24151c20 0x7f9e24072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e240cb8b0 0x7f9e24072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240c5030 con 0x7f9e24151c20 2023-10-30T12:49:14.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e24157830 con 0x7f9e240cb8b0 2023-10-30T12:49:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e29017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24157b10 con 0x7f9e24151840 2023-10-30T12:49:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e1bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e24151c20 0x7f9e24072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e1bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9e24151c20 0x7f9e24072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56284/0 (socket says 172.21.15.8:56284) 2023-10-30T12:49:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.926+0000 7f9e1bfff640 1 -- 172.21.15.8:0/1892127813 learned_addr learned my addr 172.21.15.8:0/1892127813 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e2359e640 1 --2- 172.21.15.8:0/1892127813 >> v2:172.21.15.46:3300/0 conn(0x7f9e240cb8b0 0x7f9e24072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e22d9d640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e24151840 0x7f9e240c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 -- 172.21.15.8:0/1892127813 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e24151840 msgr2=0x7f9e240c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e24151840 0x7f9e240c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 -- 172.21.15.8:0/1892127813 >> v2:172.21.15.46:3300/0 conn(0x7f9e240cb8b0 msgr2=0x7f9e24072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 --2- 172.21.15.8:0/1892127813 >> v2:172.21.15.46:3300/0 conn(0x7f9e240cb8b0 0x7f9e24072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 -- 172.21.15.8:0/1892127813 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24157df0 con 0x7f9e24151c20 2023-10-30T12:49:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e1bfff640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e24072000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cb820 tx=0x7f9e0c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13b2dd803938b6c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e2259c640 1 -- 172.21.15.8:0/1892127813 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0a89b0 con 0x7f9e24151c20 2023-10-30T12:49:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e2259c640 1 -- 172.21.15.8:0/1892127813 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0a8b10 con 0x7f9e24151c20 2023-10-30T12:49:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e2259c640 1 -- 172.21.15.8:0/1892127813 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0a8ea0 con 0x7f9e24151c20 2023-10-30T12:49:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e29017640 1 -- 172.21.15.8:0/1892127813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 msgr2=0x7f9e24072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.930+0000 7f9e29017640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e24072000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cb820 tx=0x7f9e0c01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- 172.21.15.8:0/1892127813 shutdown_connections 2023-10-30T12:49:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- 172.21.15.8:0/1892127813 >> v2:172.21.15.46:3300/0 conn(0x7f9e240cb8b0 0x7f9e24072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e24072000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- 172.21.15.8:0/1892127813 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e24151840 0x7f9e240c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- 172.21.15.8:0/1892127813 >> 172.21.15.8:0/1892127813 conn(0x7f9e240cac70 msgr2=0x7f9e240cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- 172.21.15.8:0/1892127813 shutdown_connections 2023-10-30T12:49:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- 172.21.15.8:0/1892127813 wait complete. 2023-10-30T12:49:14.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 Processor -- start 2023-10-30T12:49:14.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- start start 2023-10-30T12:49:14.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e240cb8b0 0x7f9e241f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e24151840 0x7f9e241ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241eedf0 con 0x7f9e24151c20 2023-10-30T12:49:14.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f2d10 con 0x7f9e24151840 2023-10-30T12:49:14.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e29017640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f2470 con 0x7f9e240cb8b0 2023-10-30T12:49:14.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56290/0 (socket says 172.21.15.8:56290) 2023-10-30T12:49:14.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 -- 172.21.15.8:0/949120666 learned_addr learned my addr 172.21.15.8:0/949120666 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:14.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e22d9d640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e240cb8b0 0x7f9e241f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e1bfff640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e24151840 0x7f9e241ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 -- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e240cb8b0 msgr2=0x7f9e241f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:14.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e240cb8b0 0x7f9e241f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 -- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e24151840 msgr2=0x7f9e241ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:14.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e24151840 0x7f9e241ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:14.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.934+0000 7f9e2359e640 1 -- 172.21.15.8:0/949120666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0094d0 con 0x7f9e24151c20 2023-10-30T12:49:14.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e2359e640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c007e60 tx=0x7f9e1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:14.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c014620 con 0x7f9e24151c20 2023-10-30T12:49:14.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00ee90 con 0x7f9e24151c20 2023-10-30T12:49:14.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0149e0 con 0x7f9e24151c20 2023-10-30T12:49:14.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e241f2750 con 0x7f9e24151c20 2023-10-30T12:49:14.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241eea10 con 0x7f9e24151c20 2023-10-30T12:49:14.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e241ee870 con 0x7f9e24151c20 2023-10-30T12:49:14.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e1c014b40 con 0x7f9e24151c20 2023-10-30T12:49:14.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 0x7f9e04068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:14.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 7564+0+0 (secure 0 0 0) 0x7f9e1c084d90 con 0x7f9e24151c20 2023-10-30T12:49:14.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.938+0000 7f9e22d9d640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 0x7f9e04068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:14.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.942+0000 7f9e22d9d640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 0x7f9e04068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007a60 tx=0x7f9e14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:14.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:14.942+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e241ee870 con 0x7f9e24151c20 2023-10-30T12:49:15.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.214+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e241eef30 con 0x7f9e24151c20 2023-10-30T12:49:15.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.214+0000 7f9e1b7fe640 1 -- 172.21.15.8:0/949120666 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f9e241eef30 con 0x7f9e24151c20 2023-10-30T12:49:15.224 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:15.224 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:15.225 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:15.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 msgr2=0x7f9e04068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:15.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 0x7f9e04068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007a60 tx=0x7f9e14008040 comp rx=0 tx=0).stop 2023-10-30T12:49:15.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 msgr2=0x7f9e240c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:15.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c007e60 tx=0x7f9e1c00e4b0 comp rx=0 tx=0).stop 2023-10-30T12:49:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 shutdown_connections 2023-10-30T12:49:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e040664d0 0x7f9e04068990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:15.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e24151840 0x7f9e241ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24151c20 0x7f9e240c9e80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 --2- 172.21.15.8:0/949120666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e240cb8b0 0x7f9e241f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 >> 172.21.15.8:0/949120666 conn(0x7f9e240cac70 msgr2=0x7f9e241e3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 shutdown_connections 2023-10-30T12:49:15.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:15.222+0000 7f9e29017640 1 -- 172.21.15.8:0/949120666 wait complete. 2023-10-30T12:49:15.280 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:17.281 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:17.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 Processor -- start 2023-10-30T12:49:17.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 -- start start 2023-10-30T12:49:17.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5264153bb0 0x7f52640c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5264159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52640c5f80 con 0x7f526406c420 2023-10-30T12:49:17.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526415a310 con 0x7f5264153bb0 2023-10-30T12:49:17.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f526bf0f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52640c5690 con 0x7f5264159980 2023-10-30T12:49:17.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f52694aa640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f526406c420 0x7f52640c6c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60070/0 (socket says 172.21.15.8:60070) 2023-10-30T12:49:17.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.426+0000 7f52694aa640 1 -- 172.21.15.8:0/611851721 learned_addr learned my addr 172.21.15.8:0/611851721 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:17.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f526a4ac640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5264159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969991586 0 0) 0x7f52640c5f80 con 0x7f526406c420 2023-10-30T12:49:17.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5250003530 con 0x7f526406c420 2023-10-30T12:49:17.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483938384 0 0) 0x7f5250003530 con 0x7f526406c420 2023-10-30T12:49:17.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 msgr2=0x7f5264159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:17.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5264159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 >> v2:172.21.15.46:3300/0 conn(0x7f5264153bb0 msgr2=0x7f52640c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:17.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 --2- 172.21.15.8:0/611851721 >> v2:172.21.15.46:3300/0 conn(0x7f5264153bb0 0x7f52640c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526415a580 con 0x7f526406c420 2023-10-30T12:49:17.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f5254001680 con 0x7f526406c420 2023-10-30T12:49:17.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 >> v1:172.21.15.8:6789/0 conn(0x7f526406c420 legacy=0x7f52640c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5250003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f52500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250004d10 0x7f5250005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526415a310 con 0x7f5250003fb0 2023-10-30T12:49:17.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52640c5690 con 0x7f5250004d10 2023-10-30T12:49:17.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525000bc30 con 0x7f5264159980 2023-10-30T12:49:17.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f52500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f5269cab640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5250003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 -- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 msgr2=0x7f5250003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:17.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5264159980 0x7f5250003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 -- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250004d10 msgr2=0x7f5250005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:17.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250004d10 0x7f5250005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.430+0000 7f52694aa640 1 -- 172.21.15.8:0/611851721 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250003530 con 0x7f5250003fb0 2023-10-30T12:49:17.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f52694aa640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f52500047d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f525400c240 tx=0x7f525401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f351bdbd2a927c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:17.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540a5960 con 0x7f5250003fb0 2023-10-30T12:49:17.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52540a5ac0 con 0x7f5250003fb0 2023-10-30T12:49:17.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f5268ca9640 1 -- 172.21.15.8:0/611851721 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540a5e90 con 0x7f5250003fb0 2023-10-30T12:49:17.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- 172.21.15.8:0/611851721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 msgr2=0x7f52500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 --2- 172.21.15.8:0/611851721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f52500047d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f525400c240 tx=0x7f525401cc60 comp rx=0 tx=0).stop 2023-10-30T12:49:17.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526a4ac640 1 -- 172.21.15.8:0/611851721 reap_dead start 2023-10-30T12:49:17.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- 172.21.15.8:0/611851721 shutdown_connections 2023-10-30T12:49:17.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- 172.21.15.8:0/611851721 >> 172.21.15.8:0/611851721 conn(0x7f52640cac70 msgr2=0x7f52640cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:17.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- 172.21.15.8:0/611851721 shutdown_connections 2023-10-30T12:49:17.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- 172.21.15.8:0/611851721 wait complete. 2023-10-30T12:49:17.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 Processor -- start 2023-10-30T12:49:17.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.434+0000 7f526bf0f640 1 -- start start 2023-10-30T12:49:17.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52640c6560 0x7f52641f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52641f24c0 0x7f52641418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5264148bc0 con 0x7f5250003fb0 2023-10-30T12:49:17.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52641f2940 con 0x7f52640c6560 2023-10-30T12:49:17.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5264148620 con 0x7f52641f24c0 2023-10-30T12:49:17.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56306/0 (socket says 172.21.15.8:56306) 2023-10-30T12:49:17.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 -- 172.21.15.8:0/4254901007 learned_addr learned my addr 172.21.15.8:0/4254901007 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:17.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526a4ac640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52641f24c0 0x7f52641418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f52694aa640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52640c6560 0x7f52641f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 -- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52641f24c0 msgr2=0x7f52641418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52641f24c0 0x7f52641418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 -- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52640c6560 msgr2=0x7f52641f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52640c6560 0x7f52641f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 -- 172.21.15.8:0/4254901007 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264152620 con 0x7f5250003fb0 2023-10-30T12:49:17.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f5269cab640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5260006b50 tx=0x7f526000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:17.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600096a0 con 0x7f5250003fb0 2023-10-30T12:49:17.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5260009800 con 0x7f5250003fb0 2023-10-30T12:49:17.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52641488e0 con 0x7f5250003fb0 2023-10-30T12:49:17.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260009bb0 con 0x7f5250003fb0 2023-10-30T12:49:17.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5264154770 con 0x7f5250003fb0 2023-10-30T12:49:17.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526404e170 con 0x7f5250003fb0 2023-10-30T12:49:17.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.438+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5260009d10 con 0x7f5250003fb0 2023-10-30T12:49:17.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.442+0000 7f525a7fc640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 0x7f5248068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:17.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.442+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f5260084950 con 0x7f5250003fb0 2023-10-30T12:49:17.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.442+0000 7f52694aa640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 0x7f5248068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:17.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.442+0000 7f52694aa640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 0x7f5248068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f525400c240 tx=0x7f5254001a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:17.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.446+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f52600517b0 con 0x7f5250003fb0 2023-10-30T12:49:17.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52640c4dd0 con 0x7f5250003fb0 2023-10-30T12:49:17.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f525a7fc640 1 -- 172.21.15.8:0/4254901007 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f5260054dd0 con 0x7f5250003fb0 2023-10-30T12:49:17.644 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:17.645 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:17.646 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:17.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 msgr2=0x7f5248068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 0x7f5248068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f525400c240 tx=0x7f5254001a80 comp rx=0 tx=0).stop 2023-10-30T12:49:17.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 msgr2=0x7f5264159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:17.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5260006b50 tx=0x7f526000dc60 comp rx=0 tx=0).stop 2023-10-30T12:49:17.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 shutdown_connections 2023-10-30T12:49:17.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5248066470 0x7f5248068930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52640c6560 0x7f52641f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52641f24c0 0x7f52641418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 --2- 172.21.15.8:0/4254901007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5250003fb0 0x7f5264159040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:17.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 >> 172.21.15.8:0/4254901007 conn(0x7f52640cac70 msgr2=0x7f5264150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:17.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 shutdown_connections 2023-10-30T12:49:17.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:17.638+0000 7f526bf0f640 1 -- 172.21.15.8:0/4254901007 wait complete. 2023-10-30T12:49:17.665 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:19.666 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.814+0000 7f476191d640 1 Processor -- start 2023-10-30T12:49:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.814+0000 7f476191d640 1 -- start start 2023-10-30T12:49:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f476191d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f475c153bb0 0x7f475c0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f476191d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c159980 0x7f475c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f476191d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c0c5f80 con 0x7f475c06c420 2023-10-30T12:49:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f476191d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c15a310 con 0x7f475c153bb0 2023-10-30T12:49:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f476191d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0c5690 con 0x7f475c159980 2023-10-30T12:49:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f475c153bb0 0x7f475c0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f475c153bb0 0x7f475c0c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51890/0 (socket says 172.21.15.8:51890) 2023-10-30T12:49:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 -- 172.21.15.8:0/4217166853 learned_addr learned my addr 172.21.15.8:0/4217166853 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:19.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475b7fe640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c159980 0x7f475c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 -- 172.21.15.8:0/4217166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c159980 msgr2=0x7f475c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:19.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c159980 0x7f475c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 -- 172.21.15.8:0/4217166853 >> v1:172.21.15.46:6789/0 conn(0x7f475c06c420 legacy=0x7f475c0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 -- 172.21.15.8:0/4217166853 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c15a580 con 0x7f475c153bb0 2023-10-30T12:49:19.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475affd640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c153bb0 0x7f475c0c6410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f474c009870 tx=0x7f474c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9f4f0be982cf37 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.818+0000 7f475a7fc640 1 -- 172.21.15.8:0/4217166853 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c0a8a30 con 0x7f475c153bb0 2023-10-30T12:49:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475a7fc640 1 -- 172.21.15.8:0/4217166853 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c0a8b90 con 0x7f475c153bb0 2023-10-30T12:49:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475a7fc640 1 -- 172.21.15.8:0/4217166853 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c0b0440 con 0x7f475c153bb0 2023-10-30T12:49:19.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- 172.21.15.8:0/4217166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c153bb0 msgr2=0x7f475c0c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c153bb0 0x7f475c0c6410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f474c009870 tx=0x7f474c01ed30 comp rx=0 tx=0).stop 2023-10-30T12:49:19.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- 172.21.15.8:0/4217166853 shutdown_connections 2023-10-30T12:49:19.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c159980 0x7f475c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- 172.21.15.8:0/4217166853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c153bb0 0x7f475c0c6410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- 172.21.15.8:0/4217166853 >> 172.21.15.8:0/4217166853 conn(0x7f475c0cac70 msgr2=0x7f475c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:19.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- 172.21.15.8:0/4217166853 shutdown_connections 2023-10-30T12:49:19.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- 172.21.15.8:0/4217166853 wait complete. 2023-10-30T12:49:19.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 Processor -- start 2023-10-30T12:49:19.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- start start 2023-10-30T12:49:19.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c06c420 0x7f475c1f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 0x7f475c0c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0c9b50 con 0x7f475c159980 2023-10-30T12:49:19.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1f2e20 con 0x7f475c153bb0 2023-10-30T12:49:19.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f476191d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0bfba0 con 0x7f475c06c420 2023-10-30T12:49:19.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51896/0 (socket says 172.21.15.8:51896) 2023-10-30T12:49:19.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475b7fe640 1 -- 172.21.15.8:0/1383012673 learned_addr learned my addr 172.21.15.8:0/1383012673 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:19.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f475affd640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c06c420 0x7f475c1f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.822+0000 7f4753fff640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 0x7f475c0c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 -- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c06c420 msgr2=0x7f475c1f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c06c420 0x7f475c1f2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 -- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 msgr2=0x7f475c0c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:19.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 0x7f475c0c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:19.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 -- 172.21.15.8:0/1383012673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0094d0 con 0x7f475c159980 2023-10-30T12:49:19.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f4753fff640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 0x7f475c0c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:19.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f475b7fe640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4754007e40 tx=0x7f475400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754014620 con 0x7f475c159980 2023-10-30T12:49:19.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475400ee90 con 0x7f475c159980 2023-10-30T12:49:19.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540149e0 con 0x7f475c159980 2023-10-30T12:49:19.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c0c9870 con 0x7f475c159980 2023-10-30T12:49:19.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c0bf830 con 0x7f475c159980 2023-10-30T12:49:19.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c0bfd10 con 0x7f475c159980 2023-10-30T12:49:19.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.826+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4754014b40 con 0x7f475c159980 2023-10-30T12:49:19.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.830+0000 7f4758ff9640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 0x7f473c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:19.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.830+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f4754084d20 con 0x7f475c159980 2023-10-30T12:49:19.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.830+0000 7f475affd640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 0x7f473c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:19.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.830+0000 7f475affd640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 0x7f473c068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b2000 tx=0x7f474c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:19.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:19.834+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4754051b60 con 0x7f475c159980 2023-10-30T12:49:20.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475c04e170 con 0x7f475c159980 2023-10-30T12:49:20.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f4758ff9640 1 -- 172.21.15.8:0/1383012673 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f4754055180 con 0x7f475c159980 2023-10-30T12:49:20.029 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:20.029 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:20.029 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:20.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 msgr2=0x7f473c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 0x7f473c068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b2000 tx=0x7f474c009040 comp rx=0 tx=0).stop 2023-10-30T12:49:20.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 msgr2=0x7f475c0c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:20.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4754007e40 tx=0x7f475400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 shutdown_connections 2023-10-30T12:49:20.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f473c0664a0 0x7f473c068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f475c153bb0 0x7f475c0c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f475c159980 0x7f475c0c0570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 --2- 172.21.15.8:0/1383012673 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f475c06c420 0x7f475c1f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:20.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.022+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 >> 172.21.15.8:0/1383012673 conn(0x7f475c0cac70 msgr2=0x7f475c0c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:20.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.026+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 shutdown_connections 2023-10-30T12:49:20.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:20.026+0000 7f476191d640 1 -- 172.21.15.8:0/1383012673 wait complete. 2023-10-30T12:49:20.050 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:22.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:22.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.202+0000 7f5546365640 1 Processor -- start 2023-10-30T12:49:22.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.202+0000 7f5546365640 1 -- start start 2023-10-30T12:49:22.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5546365640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5540150830 0x7f554006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5546365640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400c2310 con 0x7f55400cb7d0 2023-10-30T12:49:22.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5546365640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5540156710 con 0x7f5540070520 2023-10-30T12:49:22.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5546365640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400c4360 con 0x7f5540150830 2023-10-30T12:49:22.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553f7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f55400cb7d0 0x7f55400c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35144/0 (socket says 172.21.15.8:35144) 2023-10-30T12:49:22.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553f7fe640 1 -- 172.21.15.8:0/1901759547 learned_addr learned my addr 172.21.15.8:0/1901759547 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:22.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 --2- 172.21.15.8:0/1901759547 >> v2:172.21.15.46:3300/0 conn(0x7f5540150830 0x7f554006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131878789 0 0) 0x7f55400c2310 con 0x7f55400cb7d0 2023-10-30T12:49:22.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5528005740 con 0x7f55400cb7d0 2023-10-30T12:49:22.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1928010824 0 0) 0x7f5528005740 con 0x7f55400cb7d0 2023-10-30T12:49:22.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 >> v2:172.21.15.46:3300/0 conn(0x7f5540150830 msgr2=0x7f554006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:22.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 --2- 172.21.15.8:0/1901759547 >> v2:172.21.15.46:3300/0 conn(0x7f5540150830 0x7f554006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 >> v1:172.21.15.46:6789/0 conn(0x7f5540070520 legacy=0x7f5540160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55401569b0 con 0x7f55400cb7d0 2023-10-30T12:49:22.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f55340016a0 con 0x7f55400cb7d0 2023-10-30T12:49:22.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 >> v1:172.21.15.8:6789/0 conn(0x7f55400cb7d0 legacy=0x7f55400c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f5528005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5528006590 0x7f5528006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5528007260 0x7f552800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c4360 con 0x7f5528005b90 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552800e110 con 0x7f5528006590 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552800e2d0 con 0x7f5528007260 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f5528005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553f7fe640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5528006590 0x7f5528006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5544902640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5528007260 0x7f552800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 -- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5528007260 msgr2=0x7f552800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:22.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5528007260 0x7f552800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 -- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5528006590 msgr2=0x7f5528006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:22.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5528006590 0x7f5528006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f553ffff640 1 -- 172.21.15.8:0/1901759547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528005740 con 0x7f5528005b90 2023-10-30T12:49:22.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.206+0000 7f5544902640 1 -- 172.21.15.8:0/1901759547 reap_dead start 2023-10-30T12:49:22.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f553ffff640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f5528005f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5530009ae0 tx=0x7f553001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ee05824b9fbb79 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300a5660 con 0x7f5528005b90 2023-10-30T12:49:22.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300a57c0 con 0x7f5528005b90 2023-10-30T12:49:22.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f553effd640 1 -- 172.21.15.8:0/1901759547 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300a5b70 con 0x7f5528005b90 2023-10-30T12:49:22.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 -- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 msgr2=0x7f5528005f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f5528005f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5530009ae0 tx=0x7f553001ca70 comp rx=0 tx=0).stop 2023-10-30T12:49:22.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 -- 172.21.15.8:0/1901759547 shutdown_connections 2023-10-30T12:49:22.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 --2- 172.21.15.8:0/1901759547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f5528005f70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 -- 172.21.15.8:0/1901759547 >> 172.21.15.8:0/1901759547 conn(0x7f55400cac70 msgr2=0x7f55400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:22.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 -- 172.21.15.8:0/1901759547 shutdown_connections 2023-10-30T12:49:22.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 -- 172.21.15.8:0/1901759547 wait complete. 2023-10-30T12:49:22.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.210+0000 7f5546365640 1 Processor -- start 2023-10-30T12:49:22.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- start start 2023-10-30T12:49:22.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5540154470 0x7f55400c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55401f1fb0 0x7f55401f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5540140e70 con 0x7f5528005b90 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5540148f90 con 0x7f5540154470 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5540156120 con 0x7f55401f1fb0 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51910/0 (socket says 172.21.15.8:51910) 2023-10-30T12:49:22.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 -- 172.21.15.8:0/1963520757 learned_addr learned my addr 172.21.15.8:0/1963520757 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:22.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5544902640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55401f1fb0 0x7f55401f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553f7fe640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5540154470 0x7f55400c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 -- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55401f1fb0 msgr2=0x7f55401f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55401f1fb0 0x7f55401f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 -- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5540154470 msgr2=0x7f55400c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5540154470 0x7f55400c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 -- 172.21.15.8:0/1963520757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55401494e0 con 0x7f5528005b90 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553ffff640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f553001c8a0 tx=0x7f5530019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530004020 con 0x7f5528005b90 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300041a0 con 0x7f5528005b90 2023-10-30T12:49:22.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300b8660 con 0x7f5528005b90 2023-10-30T12:49:22.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5540149200 con 0x7f5528005b90 2023-10-30T12:49:22.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55400c64d0 con 0x7f5528005b90 2023-10-30T12:49:22.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.214+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554004e170 con 0x7f5528005b90 2023-10-30T12:49:22.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.218+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55300b4070 con 0x7f5528005b90 2023-10-30T12:49:22.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.218+0000 7f553cff9640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 0x7f552c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:22.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.218+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6604+0+0 (secure 0 0 0) 0x7f553011ed20 con 0x7f5528005b90 2023-10-30T12:49:22.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.218+0000 7f553f7fe640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 0x7f552c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:22.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.218+0000 7f553f7fe640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 0x7f552c068990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5540072400 tx=0x7f5534009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:22.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.222+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f55300ebf30 con 0x7f5528005b90 2023-10-30T12:49:22.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.414+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5540156c60 con 0x7f5528005b90 2023-10-30T12:49:22.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.414+0000 7f553cff9640 1 -- 172.21.15.8:0/1963520757 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f55300b2090 con 0x7f5528005b90 2023-10-30T12:49:22.423 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:22.423 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:22.423 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:22.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 msgr2=0x7f552c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 0x7f552c068990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5540072400 tx=0x7f5534009040 comp rx=0 tx=0).stop 2023-10-30T12:49:22.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 msgr2=0x7f554014a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:22.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f553001c8a0 tx=0x7f5530019810 comp rx=0 tx=0).stop 2023-10-30T12:49:22.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 shutdown_connections 2023-10-30T12:49:22.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f552c0664d0 0x7f552c068990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5540154470 0x7f55400c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55401f1fb0 0x7f55401f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 --2- 172.21.15.8:0/1963520757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5528005b90 0x7f554014a2e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:22.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 >> 172.21.15.8:0/1963520757 conn(0x7f55400cac70 msgr2=0x7f554014bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:22.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 shutdown_connections 2023-10-30T12:49:22.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:22.418+0000 7f5546365640 1 -- 172.21.15.8:0/1963520757 wait complete. 2023-10-30T12:49:22.444 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:24.445 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:24.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 Processor -- start 2023-10-30T12:49:24.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 -- start start 2023-10-30T12:49:24.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415cf90 0x7f27840c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2784159f20 con 0x7f27840c4a10 2023-10-30T12:49:24.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840c47e0 con 0x7f2784153c80 2023-10-30T12:49:24.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27898d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278415dec0 con 0x7f278415cf90 2023-10-30T12:49:24.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f277a7fc640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f27840c4a10 0x7f27840ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35158/0 (socket says 172.21.15.8:35158) 2023-10-30T12:49:24.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f277a7fc640 1 -- 172.21.15.8:0/828022260 learned_addr learned my addr 172.21.15.8:0/828022260 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:24.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415cf90 0x7f27840c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582846085 0 0) 0x7f2784159f20 con 0x7f27840c4a10 2023-10-30T12:49:24.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003530 con 0x7f27840c4a10 2023-10-30T12:49:24.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132763753 0 0) 0x7f27840c47e0 con 0x7f2784153c80 2023-10-30T12:49:24.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2784159f20 con 0x7f2784153c80 2023-10-30T12:49:24.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3410190294 0 0) 0x7f2770003530 con 0x7f27840c4a10 2023-10-30T12:49:24.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415cf90 msgr2=0x7f27840c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:24.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415cf90 0x7f27840c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 >> v1:172.21.15.46:6789/0 conn(0x7f2784153c80 legacy=0x7f278415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278415d620 con 0x7f27840c4a10 2023-10-30T12:49:24.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f276c001a00 con 0x7f27840c4a10 2023-10-30T12:49:24.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 >> v1:172.21.15.8:6789/0 conn(0x7f27840c4a10 legacy=0x7f27840ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770003b40 0x7f2770003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f27700047d0 0x7f2770004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2770005190 0x7f277000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278415dec0 con 0x7f2770005190 2023-10-30T12:49:24.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277000bf00 con 0x7f2770003b40 2023-10-30T12:49:24.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.594+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277000c0c0 con 0x7f27700047d0 2023-10-30T12:49:24.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415cf90 0x7f27840c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:24.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2770005190 0x7f277000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f277a7fc640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f27700047d0 0x7f2770004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f2782ffd640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770003b40 0x7f2770003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 -- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f27700047d0 msgr2=0x7f2770004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:24.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f27700047d0 0x7f2770004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 -- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770003b40 msgr2=0x7f2770003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:24.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770003b40 0x7f2770003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 -- 172.21.15.8:0/828022260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770003530 con 0x7f2770005190 2023-10-30T12:49:24.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27837fe640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2770005190 0x7f277000b990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277c009b20 tx=0x7f277c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb57f406494efccd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c004030 con 0x7f2770005190 2023-10-30T12:49:24.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c004190 con 0x7f2770005190 2023-10-30T12:49:24.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.598+0000 7f27827fc640 1 -- 172.21.15.8:0/828022260 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c004520 con 0x7f2770005190 2023-10-30T12:49:24.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- 172.21.15.8:0/828022260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2770005190 msgr2=0x7f277000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 --2- 172.21.15.8:0/828022260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2770005190 0x7f277000b990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277c009b20 tx=0x7f277c01ca90 comp rx=0 tx=0).stop 2023-10-30T12:49:24.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27837fe640 1 -- 172.21.15.8:0/828022260 reap_dead start 2023-10-30T12:49:24.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- 172.21.15.8:0/828022260 shutdown_connections 2023-10-30T12:49:24.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- 172.21.15.8:0/828022260 >> 172.21.15.8:0/828022260 conn(0x7f27840715c0 msgr2=0x7f27840719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:24.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- 172.21.15.8:0/828022260 shutdown_connections 2023-10-30T12:49:24.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- 172.21.15.8:0/828022260 wait complete. 2023-10-30T12:49:24.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 Processor -- start 2023-10-30T12:49:24.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- start start 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 0x7f27840c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415b9d0 0x7f278414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784157da0 con 0x7f2784159640 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278415be50 con 0x7f2770005190 2023-10-30T12:49:24.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27898d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278414d500 con 0x7f278415b9d0 2023-10-30T12:49:24.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f277a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f277a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51924/0 (socket says 172.21.15.8:51924) 2023-10-30T12:49:24.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f277a7fc640 1 -- 172.21.15.8:0/3809492035 learned_addr learned my addr 172.21.15.8:0/3809492035 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:24.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f2782ffd640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 0x7f27840c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.602+0000 7f27837fe640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415b9d0 0x7f278414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 -- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415b9d0 msgr2=0x7f278414c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415b9d0 0x7f278414c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 -- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 msgr2=0x7f27840c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 0x7f27840c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 -- 172.21.15.8:0/3809492035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278414ac90 con 0x7f2784159640 2023-10-30T12:49:24.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2782ffd640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 0x7f27840c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:24.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f277a7fc640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f276c001790 tx=0x7f276c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00aca0 con 0x7f2784159640 2023-10-30T12:49:24.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c00ae00 con 0x7f2784159640 2023-10-30T12:49:24.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c013630 con 0x7f2784159640 2023-10-30T12:49:24.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278414a9b0 con 0x7f2784159640 2023-10-30T12:49:24.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278415ac90 con 0x7f2784159640 2023-10-30T12:49:24.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278404e170 con 0x7f2784159640 2023-10-30T12:49:24.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f276c013790 con 0x7f2784159640 2023-10-30T12:49:24.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 0x7f2764068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:24.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6604+0+0 (secure 0 0 0) 0x7f276c084ec0 con 0x7f2784159640 2023-10-30T12:49:24.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.606+0000 7f2782ffd640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 0x7f2764068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:24.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.610+0000 7f2782ffd640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 0x7f2764068990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2774007fd0 tx=0x7f2774009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:24.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.610+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f276c051f20 con 0x7f2784159640 2023-10-30T12:49:24.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.802+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27840713a0 con 0x7f2784159640 2023-10-30T12:49:24.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f2780ff9640 1 -- 172.21.15.8:0/3809492035 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f276c017070 con 0x7f2784159640 2023-10-30T12:49:24.811 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:24.811 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:47:59.157117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":41,"addr":"172.21.15.8:6838/2992596118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":2992596118},{"type":"v1","addr":"172.21.15.8:6838","nonce":2992596118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:24.811 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 20 2023-10-30T12:49:24.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 msgr2=0x7f2764068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 0x7f2764068990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2774007fd0 tx=0x7f2774009040 comp rx=0 tx=0).stop 2023-10-30T12:49:24.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 msgr2=0x7f278415b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:24.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f276c001790 tx=0x7f276c007bd0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 shutdown_connections 2023-10-30T12:49:24.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f27640664d0 0x7f2764068990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f278415b9d0 0x7f278414c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2784159640 0x7f278415b490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 --2- 172.21.15.8:0/3809492035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2770005190 0x7f27840c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:24.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 >> 172.21.15.8:0/3809492035 conn(0x7f27840715c0 msgr2=0x7f2784151e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:24.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 shutdown_connections 2023-10-30T12:49:24.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:24.806+0000 7f27898d6640 1 -- 172.21.15.8:0/3809492035 wait complete. 2023-10-30T12:49:24.832 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:49:26.833 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:26.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 Processor -- start 2023-10-30T12:49:26.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 -- start start 2023-10-30T12:49:26.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9f50065ba0 0x7f9f5015a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:26.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5006e390 con 0x7f9f50071340 2023-10-30T12:49:26.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f500bec10 con 0x7f9f50153c10 2023-10-30T12:49:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f5826b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f50072390 con 0x7f9f50065ba0 2023-10-30T12:49:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f56808640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9f50065ba0 0x7f9f5015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f55806640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f9f50071340 0x7f9f50159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35168/0 (socket says 172.21.15.8:35168) 2023-10-30T12:49:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f56808640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f9f50065ba0 0x7f9f5015a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51934/0 (socket says 172.21.15.8:51934) 2023-10-30T12:49:26.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f55806640 1 -- 172.21.15.8:0/4102043587 learned_addr learned my addr 172.21.15.8:0/4102043587 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f56808640 1 -- 172.21.15.8:0/4102043587 >> v1:172.21.15.46:6789/0 conn(0x7f9f50153c10 legacy=0x7f9f50070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f56808640 1 -- 172.21.15.8:0/4102043587 >> v1:172.21.15.8:6789/0 conn(0x7f9f50071340 legacy=0x7f9f50159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:26.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.982+0000 7f9f56808640 1 -- 172.21.15.8:0/4102043587 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5006e5e0 con 0x7f9f50065ba0 2023-10-30T12:49:26.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f56808640 1 --2- 172.21.15.8:0/4102043587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f5015a080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009d90 tx=0x7f9f4801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f375262ad268d54b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:26.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f55005640 1 -- 172.21.15.8:0/4102043587 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480a6660 con 0x7f9f50065ba0 2023-10-30T12:49:26.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f55005640 1 -- 172.21.15.8:0/4102043587 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f480a67c0 con 0x7f9f50065ba0 2023-10-30T12:49:26.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f55005640 1 -- 172.21.15.8:0/4102043587 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480a6b50 con 0x7f9f50065ba0 2023-10-30T12:49:26.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 -- 172.21.15.8:0/4102043587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 msgr2=0x7f9f5015a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:26.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 --2- 172.21.15.8:0/4102043587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f5015a080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009d90 tx=0x7f9f4801c8c0 comp rx=0 tx=0).stop 2023-10-30T12:49:26.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 -- 172.21.15.8:0/4102043587 shutdown_connections 2023-10-30T12:49:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 --2- 172.21.15.8:0/4102043587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f5015a080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 -- 172.21.15.8:0/4102043587 >> 172.21.15.8:0/4102043587 conn(0x7f9f500c7f60 msgr2=0x7f9f500c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 -- 172.21.15.8:0/4102043587 shutdown_connections 2023-10-30T12:49:26.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.986+0000 7f9f5826b640 1 -- 172.21.15.8:0/4102043587 wait complete. 2023-10-30T12:49:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 Processor -- start 2023-10-30T12:49:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- start start 2023-10-30T12:49:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:26.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f50071340 0x7f9f5013d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9f50153c10 0x7f9f5014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f501f5d10 con 0x7f9f50065ba0 2023-10-30T12:49:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5014dee0 con 0x7f9f50071340 2023-10-30T12:49:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5013cc70 con 0x7f9f50153c10 2023-10-30T12:49:26.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51942/0 (socket says 172.21.15.8:51942) 2023-10-30T12:49:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 -- 172.21.15.8:0/2412740102 learned_addr learned my addr 172.21.15.8:0/2412740102 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f55806640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f50071340 0x7f9f5013d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56808640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9f50153c10 0x7f9f5014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:26.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 -- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9f50153c10 msgr2=0x7f9f5014c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9f50153c10 0x7f9f5014c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 -- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f50071340 msgr2=0x7f9f5013d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f50071340 0x7f9f5013d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 -- 172.21.15.8:0/2412740102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48009a70 con 0x7f9f50065ba0 2023-10-30T12:49:27.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f56007640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c3c0 tx=0x7f9f4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0052d0 con 0x7f9f50065ba0 2023-10-30T12:49:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c005430 con 0x7f9f50065ba0 2023-10-30T12:49:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003840 con 0x7f9f50065ba0 2023-10-30T12:49:27.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f501f5a30 con 0x7f9f50065ba0 2023-10-30T12:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.990+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5013c190 con 0x7f9f50065ba0 2023-10-30T12:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5004e170 con 0x7f9f50065ba0 2023-10-30T12:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f4c0039a0 con 0x7f9f50065ba0 2023-10-30T12:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f46ffd640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 0x7f9f34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6604+0+0 (secure 0 0 0) 0x7f9f4c083760 con 0x7f9f50065ba0 2023-10-30T12:49:27.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f55806640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 0x7f9f34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.994+0000 7f9f55806640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 0x7f9f34068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f500be7c0 tx=0x7f9f40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:26.998+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f4c050850 con 0x7f9f50065ba0 2023-10-30T12:49:27.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.194+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f50070dd0 con 0x7f9f50065ba0 2023-10-30T12:49:27.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f46ffd640 1 -- 172.21.15.8:0/2412740102 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5727 (secure 0 0 0) 0x7f9f4c053e70 con 0x7f9f50065ba0 2023-10-30T12:49:27.203 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:27.204 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:27.084945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 msgr2=0x7f9f34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 0x7f9f34068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f500be7c0 tx=0x7f9f40009040 comp rx=0 tx=0).stop 2023-10-30T12:49:27.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 msgr2=0x7f9f50141720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c3c0 tx=0x7f9f4c00c880 comp rx=0 tx=0).stop 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 shutdown_connections 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9f340664a0 0x7f9f34068960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f50071340 0x7f9f5013d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9f50153c10 0x7f9f5014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 --2- 172.21.15.8:0/2412740102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50065ba0 0x7f9f50141720 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 >> 172.21.15.8:0/2412740102 conn(0x7f9f500c7f60 msgr2=0x7f9f500ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 shutdown_connections 2023-10-30T12:49:27.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.198+0000 7f9f5826b640 1 -- 172.21.15.8:0/2412740102 wait complete. 2023-10-30T12:49:27.224 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-30T12:49:27.224 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:49:27.224 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:27.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 Processor -- start 2023-10-30T12:49:27.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 -- start start 2023-10-30T12:49:27.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fea84151c20 0x7fea840c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea840cb8b0 0x7fea8406b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840c5030 con 0x7fea84151840 2023-10-30T12:49:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea84157830 con 0x7fea84151c20 2023-10-30T12:49:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8c7f5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea84157b10 con 0x7fea840cb8b0 2023-10-30T12:49:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fea84151c20 0x7fea840c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fea84151c20 0x7fea840c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51946/0 (socket says 172.21.15.8:51946) 2023-10-30T12:49:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 -- 172.21.15.8:0/890146379 learned_addr learned my addr 172.21.15.8:0/890146379 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea8ad92640 1 --2- 172.21.15.8:0/890146379 >> v2:172.21.15.46:3300/0 conn(0x7fea840cb8b0 0x7fea8406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 -- 172.21.15.8:0/890146379 >> v2:172.21.15.46:3300/0 conn(0x7fea840cb8b0 msgr2=0x7fea8406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 --2- 172.21.15.8:0/890146379 >> v2:172.21.15.46:3300/0 conn(0x7fea840cb8b0 0x7fea8406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 -- 172.21.15.8:0/890146379 >> v1:172.21.15.46:6789/0 conn(0x7fea84151840 legacy=0x7fea84072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.362+0000 7fea89d90640 1 -- 172.21.15.8:0/890146379 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84157df0 con 0x7fea84151c20 2023-10-30T12:49:27.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea89d90640 1 --2- 172.21.15.8:0/890146379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151c20 0x7fea840c49a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea840c6cb0 tx=0x7fea7401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a8fb0fe9532c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8958f640 1 -- 172.21.15.8:0/890146379 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740a89f0 con 0x7fea84151c20 2023-10-30T12:49:27.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8958f640 1 -- 172.21.15.8:0/890146379 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740a8b50 con 0x7fea84151c20 2023-10-30T12:49:27.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8958f640 1 -- 172.21.15.8:0/890146379 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740b0440 con 0x7fea84151c20 2023-10-30T12:49:27.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 -- 172.21.15.8:0/890146379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151c20 msgr2=0x7fea840c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/890146379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151c20 0x7fea840c49a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea840c6cb0 tx=0x7fea7401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 -- 172.21.15.8:0/890146379 shutdown_connections 2023-10-30T12:49:27.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/890146379 >> v2:172.21.15.46:3300/0 conn(0x7fea840cb8b0 0x7fea8406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/890146379 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151c20 0x7fea840c49a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 -- 172.21.15.8:0/890146379 >> 172.21.15.8:0/890146379 conn(0x7fea840cac70 msgr2=0x7fea840cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:27.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 -- 172.21.15.8:0/890146379 shutdown_connections 2023-10-30T12:49:27.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.366+0000 7fea8c7f5640 1 -- 172.21.15.8:0/890146379 wait complete. 2023-10-30T12:49:27.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 Processor -- start 2023-10-30T12:49:27.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- start start 2023-10-30T12:49:27.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea840cb8b0 0x7fea84070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fea84151c20 0x7fea841eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840ca420 con 0x7fea84151840 2023-10-30T12:49:27.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840c9b10 con 0x7fea840cb8b0 2023-10-30T12:49:27.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840712b0 con 0x7fea84151c20 2023-10-30T12:49:27.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51952/0 (socket says 172.21.15.8:51952) 2023-10-30T12:49:27.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 -- 172.21.15.8:0/517043641 learned_addr learned my addr 172.21.15.8:0/517043641 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:27.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8a591640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea840cb8b0 0x7fea84070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8ad92640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fea84151c20 0x7fea841eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 -- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fea84151c20 msgr2=0x7fea841eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fea84151c20 0x7fea841eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 -- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea840cb8b0 msgr2=0x7fea84070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea840cb8b0 0x7fea84070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 -- 172.21.15.8:0/517043641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740094d0 con 0x7fea84151840 2023-10-30T12:49:27.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea89d90640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea7401b900 tx=0x7fea7401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740ba600 con 0x7fea84151840 2023-10-30T12:49:27.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740ba760 con 0x7fea84151840 2023-10-30T12:49:27.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea84071590 con 0x7fea84151840 2023-10-30T12:49:27.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740bab10 con 0x7fea84151840 2023-10-30T12:49:27.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.370+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840ca040 con 0x7fea84151840 2023-10-30T12:49:27.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8404e170 con 0x7fea84151840 2023-10-30T12:49:27.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fea740bac70 con 0x7fea84151840 2023-10-30T12:49:27.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea7affd640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 0x7fea68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:27.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fea74122740 con 0x7fea84151840 2023-10-30T12:49:27.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea8a591640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 0x7fea68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:27.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.374+0000 7fea8a591640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 0x7fea68068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:27.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.378+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fea740ef7d0 con 0x7fea84151840 2023-10-30T12:49:27.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.566+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea84066510 con 0x7fea84151840 2023-10-30T12:49:27.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.566+0000 7fea7affd640 1 -- 172.21.15.8:0/517043641 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5727 (secure 0 0 0) 0x7fea740f2df0 con 0x7fea84151840 2023-10-30T12:49:27.575 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:27.575 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:27.084945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2023-10-30T12:49:27.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 msgr2=0x7fea68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 0x7fea68068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).stop 2023-10-30T12:49:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 msgr2=0x7fea8406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea7401b900 tx=0x7fea7401ba90 comp rx=0 tx=0).stop 2023-10-30T12:49:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 shutdown_connections 2023-10-30T12:49:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fea680664a0 0x7fea68068960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fea84151c20 0x7fea841eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea84151840 0x7fea8406b760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 --2- 172.21.15.8:0/517043641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea840cb8b0 0x7fea84070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 >> 172.21.15.8:0/517043641 conn(0x7fea840cac70 msgr2=0x7fea840733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 shutdown_connections 2023-10-30T12:49:27.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:27.570+0000 7fea8c7f5640 1 -- 172.21.15.8:0/517043641 wait complete. 2023-10-30T12:49:27.596 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4276": { "addr": "172.21.15.46:6833/3888673612", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 3888673612, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 3888673612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4276, "incarnation": 9, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:49:27.084945+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 4276, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:49:29.598 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 Processor -- start 2023-10-30T12:49:29.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 -- start start 2023-10-30T12:49:29.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93ac0c6320 0x7f93ac06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac0c55f0 con 0x7f93ac14fc50 2023-10-30T12:49:29.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac1567b0 con 0x7f93ac06be90 2023-10-30T12:49:29.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93b0b65640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0c5f00 con 0x7f93ac0c6320 2023-10-30T12:49:29.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93aa59c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f93ac14fc50 0x7f93ac070e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35826/0 (socket says 172.21.15.8:35826) 2023-10-30T12:49:29.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.746+0000 7f93aa59c640 1 -- 172.21.15.8:0/3874110695 learned_addr learned my addr 172.21.15.8:0/3874110695 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:29.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a3fff640 1 --2- 172.21.15.8:0/3874110695 >> v2:172.21.15.46:3300/0 conn(0x7f93ac0c6320 0x7f93ac06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429654714 0 0) 0x7f93ac0c55f0 con 0x7f93ac14fc50 2023-10-30T12:49:29.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9398003530 con 0x7f93ac14fc50 2023-10-30T12:49:29.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472459046 0 0) 0x7f93ac1567b0 con 0x7f93ac06be90 2023-10-30T12:49:29.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ac0c55f0 con 0x7f93ac06be90 2023-10-30T12:49:29.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150776793 0 0) 0x7f9398003530 con 0x7f93ac14fc50 2023-10-30T12:49:29.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 >> v2:172.21.15.46:3300/0 conn(0x7f93ac0c6320 msgr2=0x7f93ac06fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 --2- 172.21.15.8:0/3874110695 >> v2:172.21.15.46:3300/0 conn(0x7f93ac0c6320 0x7f93ac06fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 >> v1:172.21.15.46:6789/0 conn(0x7f93ac06be90 legacy=0x7f93ac155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac156a70 con 0x7f93ac14fc50 2023-10-30T12:49:29.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f939c001680 con 0x7f93ac14fc50 2023-10-30T12:49:29.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 >> v1:172.21.15.8:6789/0 conn(0x7f93ac14fc50 legacy=0x7f93ac070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 0x7f9398003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93980047d0 0x7f9398004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398005190 0x7f939800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0c5f00 con 0x7f9398003b10 2023-10-30T12:49:29.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939800c0c0 con 0x7f9398005190 2023-10-30T12:49:29.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939800c280 con 0x7f93980047d0 2023-10-30T12:49:29.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 0x7f9398003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93a3fff640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93980047d0 0x7f9398004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aad9d640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398005190 0x7f939800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 -- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93980047d0 msgr2=0x7f9398004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93980047d0 0x7f9398004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 -- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398005190 msgr2=0x7f939800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398005190 0x7f939800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aa59c640 1 -- 172.21.15.8:0/3874110695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398003530 con 0x7f9398003b10 2023-10-30T12:49:29.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.750+0000 7f93aad9d640 1 -- 172.21.15.8:0/3874110695 reap_dead start 2023-10-30T12:49:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 0x7f9398003f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c160 tx=0x7f939c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72045845321028 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0058e0 con 0x7f9398003b10 2023-10-30T12:49:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c005a40 con 0x7f9398003b10 2023-10-30T12:49:29.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93a9d9b640 1 -- 172.21.15.8:0/3874110695 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c005df0 con 0x7f9398003b10 2023-10-30T12:49:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 msgr2=0x7f9398003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 0x7f9398003f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c160 tx=0x7f939c01cbc0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- 172.21.15.8:0/3874110695 shutdown_connections 2023-10-30T12:49:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3874110695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9398003b10 0x7f9398003f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- 172.21.15.8:0/3874110695 >> 172.21.15.8:0/3874110695 conn(0x7f93ac0cac70 msgr2=0x7f93ac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- 172.21.15.8:0/3874110695 shutdown_connections 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- 172.21.15.8:0/3874110695 wait complete. 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 Processor -- start 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.754+0000 7f93b0b65640 1 -- start start 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 0x7f93ac149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93ac1ef180 0x7f93ac143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac149690 con 0x7f93ac159c10 2023-10-30T12:49:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac1ee750 con 0x7f9398003b10 2023-10-30T12:49:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac1f1e10 con 0x7f93ac1ef180 2023-10-30T12:49:29.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57766/0 (socket says 172.21.15.8:57766) 2023-10-30T12:49:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 -- 172.21.15.8:0/3490652740 learned_addr learned my addr 172.21.15.8:0/3490652740 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 0x7f93ac149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93aad9d640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93ac1ef180 0x7f93ac143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 -- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93ac1ef180 msgr2=0x7f93ac143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93ac1ef180 0x7f93ac143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 -- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 msgr2=0x7f93ac149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 0x7f93ac149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 -- 172.21.15.8:0/3490652740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac1eebe0 con 0x7f93ac159c10 2023-10-30T12:49:29.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 0x7f93ac149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a3fff640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9394011560 tx=0x7f9394011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394004440 con 0x7f93ac159c10 2023-10-30T12:49:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394003620 con 0x7f93ac159c10 2023-10-30T12:49:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac1f20d0 con 0x7f93ac159c10 2023-10-30T12:49:29.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940039b0 con 0x7f93ac159c10 2023-10-30T12:49:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac1ee100 con 0x7f93ac159c10 2023-10-30T12:49:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac04e170 con 0x7f93ac159c10 2023-10-30T12:49:29.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.758+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9394003b10 con 0x7f93ac159c10 2023-10-30T12:49:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.762+0000 7f93a2ffd640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 0x7f938c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.762+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f9394084490 con 0x7f93ac159c10 2023-10-30T12:49:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.762+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 0x7f938c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.762+0000 7f93aa59c640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 0x7f938c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f939c0aa000 tx=0x7f939c001a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.766+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9394051490 con 0x7f93ac159c10 2023-10-30T12:49:29.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93ac0c7300 con 0x7f93ac159c10 2023-10-30T12:49:29.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93a2ffd640 1 -- 172.21.15.8:0/3490652740 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5727 (secure 0 0 0) 0x7f9394054ab0 con 0x7f93ac159c10 2023-10-30T12:49:29.961 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:29.961 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:27.084945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:29.962 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2023-10-30T12:49:29.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 msgr2=0x7f938c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 0x7f938c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f939c0aa000 tx=0x7f939c001a00 comp rx=0 tx=0).stop 2023-10-30T12:49:29.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 msgr2=0x7f93ac0c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:29.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9394011560 tx=0x7f9394011a20 comp rx=0 tx=0).stop 2023-10-30T12:49:29.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 shutdown_connections 2023-10-30T12:49:29.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f938c0664a0 0x7f938c068960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.954+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f93ac1ef180 0x7f93ac143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.958+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ac159c10 0x7f93ac0c3d20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.958+0000 7f93b0b65640 1 --2- 172.21.15.8:0/3490652740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9398003b10 0x7f93ac149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:29.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.958+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 >> 172.21.15.8:0/3490652740 conn(0x7f93ac0cac70 msgr2=0x7f93ac0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:29.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.958+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 shutdown_connections 2023-10-30T12:49:29.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:29.958+0000 7f93b0b65640 1 -- 172.21.15.8:0/3490652740 wait complete. 2023-10-30T12:49:29.982 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:31.983 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:32.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 Processor -- start 2023-10-30T12:49:32.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 -- start start 2023-10-30T12:49:32.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 0x7fc8a415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a4159f20 con 0x7fc8a40c42a0 2023-10-30T12:49:32.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a415d620 con 0x7fc8a4153c80 2023-10-30T12:49:32.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a8f65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a415dec0 con 0x7fc8a40c4a10 2023-10-30T12:49:32.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a2d9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fc8a40c42a0 0x7fc8a40ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35830/0 (socket says 172.21.15.8:35830) 2023-10-30T12:49:32.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/78712342 learned_addr learned my addr 172.21.15.8:0/78712342 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:32.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc89bfff640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 0x7fc8a415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224695635 0 0) 0x7fc8a4159f20 con 0x7fc8a40c42a0 2023-10-30T12:49:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc890003530 con 0x7fc8a40c42a0 2023-10-30T12:49:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657616586 0 0) 0x7fc8a415d620 con 0x7fc8a4153c80 2023-10-30T12:49:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a4159f20 con 0x7fc8a4153c80 2023-10-30T12:49:32.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773201623 0 0) 0x7fc890003530 con 0x7fc8a40c42a0 2023-10-30T12:49:32.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 msgr2=0x7fc8a415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:32.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 0x7fc8a415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 >> v1:172.21.15.46:6789/0 conn(0x7fc8a4153c80 legacy=0x7fc8a415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a415d900 con 0x7fc8a40c42a0 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fc89c001980 con 0x7fc8a40c42a0 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 >> v1:172.21.15.8:6789/0 conn(0x7fc8a40c42a0 legacy=0x7fc8a40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc890003b40 0x7fc890003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8900047d0 0x7fc890004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 0x7fc890005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a415dec0 con 0x7fc8900047d0 2023-10-30T12:49:32.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89000bb20 con 0x7fc890003b40 2023-10-30T12:49:32.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.138+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89000bcb0 con 0x7fc8a40c4a10 2023-10-30T12:49:32.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8900047d0 0x7fc890004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc8a259c640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc890003b40 0x7fc890003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 -- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 msgr2=0x7fc890005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:32.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a40c4a10 0x7fc890005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 -- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc890003b40 msgr2=0x7fc890003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:32.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc890003b40 0x7fc890003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 -- 172.21.15.8:0/78712342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890003530 con 0x7fc8900047d0 2023-10-30T12:49:32.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc89bfff640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8900047d0 0x7fc890004c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ba00 tx=0x7fc88c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f193aee7e73510ff server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:32.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0a9660 con 0x7fc8900047d0 2023-10-30T12:49:32.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c0a97c0 con 0x7fc8900047d0 2023-10-30T12:49:32.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.142+0000 7fc8a1d9b640 1 -- 172.21.15.8:0/78712342 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0a9b70 con 0x7fc8900047d0 2023-10-30T12:49:32.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- 172.21.15.8:0/78712342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8900047d0 msgr2=0x7fc890004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/78712342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8900047d0 0x7fc890004c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ba00 tx=0x7fc88c01c680 comp rx=0 tx=0).stop 2023-10-30T12:49:32.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/78712342 reap_dead start 2023-10-30T12:49:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- 172.21.15.8:0/78712342 shutdown_connections 2023-10-30T12:49:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- 172.21.15.8:0/78712342 >> 172.21.15.8:0/78712342 conn(0x7fc8a40715c0 msgr2=0x7fc8a40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- 172.21.15.8:0/78712342 shutdown_connections 2023-10-30T12:49:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- 172.21.15.8:0/78712342 wait complete. 2023-10-30T12:49:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 Processor -- start 2023-10-30T12:49:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- start start 2023-10-30T12:49:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8900047d0 0x7fc8a41ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41f28c0 0x7fc8a4158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a414dab0 con 0x7fc8a4157810 2023-10-30T12:49:32.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40c8a90 con 0x7fc8a41f28c0 2023-10-30T12:49:32.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a8f65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40c8d70 con 0x7fc8900047d0 2023-10-30T12:49:32.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57778/0 (socket says 172.21.15.8:57778) 2023-10-30T12:49:32.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/1583098641 learned_addr learned my addr 172.21.15.8:0/1583098641 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc8a259c640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8900047d0 0x7fc8a41ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.146+0000 7fc89bfff640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41f28c0 0x7fc8a4158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8900047d0 msgr2=0x7fc8a41ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8900047d0 0x7fc8a41ef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41f28c0 msgr2=0x7fc8a4158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41f28c0 0x7fc8a4158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 -- 172.21.15.8:0/1583098641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40c9050 con 0x7fc8a4157810 2023-10-30T12:49:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a2d9d640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004c00 tx=0x7fc89c00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00da80 con 0x7fc8a4157810 2023-10-30T12:49:32.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c00dbe0 con 0x7fc8a4157810 2023-10-30T12:49:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00a030 con 0x7fc8a4157810 2023-10-30T12:49:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a41eecc0 con 0x7fc8a4157810 2023-10-30T12:49:32.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a41ee970 con 0x7fc8a4157810 2023-10-30T12:49:32.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a404e170 con 0x7fc8a4157810 2023-10-30T12:49:32.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc89c00a190 con 0x7fc8a4157810 2023-10-30T12:49:32.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 0x7fc884068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:32.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fc89c086a60 con 0x7fc8a4157810 2023-10-30T12:49:32.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.150+0000 7fc8a259c640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 0x7fc884068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:32.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.154+0000 7fc8a259c640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 0x7fc884068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc894007af0 tx=0x7fc894009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:32.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.154+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc89c053ba0 con 0x7fc8a4157810 2023-10-30T12:49:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.350+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8a40b1b20 con 0x7fc8a4157810 2023-10-30T12:49:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.350+0000 7fc89affd640 1 -- 172.21.15.8:0/1583098641 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5727 (secure 0 0 0) 0x7fc89c01b020 con 0x7fc8a4157810 2023-10-30T12:49:32.358 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:32.358 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:27.084945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2023-10-30T12:49:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 msgr2=0x7fc884068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 0x7fc884068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc894007af0 tx=0x7fc894009040 comp rx=0 tx=0).stop 2023-10-30T12:49:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 msgr2=0x7fc8a4157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004c00 tx=0x7fc89c00ff00 comp rx=0 tx=0).stop 2023-10-30T12:49:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 shutdown_connections 2023-10-30T12:49:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664d0 0x7fc884068990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41f28c0 0x7fc8a4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a4157810 0x7fc8a4157c90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 --2- 172.21.15.8:0/1583098641 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8900047d0 0x7fc8a41ef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 >> 172.21.15.8:0/1583098641 conn(0x7fc8a40715c0 msgr2=0x7fc8a40c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 shutdown_connections 2023-10-30T12:49:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:32.354+0000 7fc8a8f65640 1 -- 172.21.15.8:0/1583098641 wait complete. 2023-10-30T12:49:32.379 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:34.380 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:34.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.526+0000 7fd295ed6640 1 Processor -- start 2023-10-30T12:49:34.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 -- start start 2023-10-30T12:49:34.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd290151840 0x7fd2900c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd2900cb8b0 0x7fd29006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900c5030 con 0x7fd290151c20 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd290157830 con 0x7fd2900cb8b0 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd295ed6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290157b10 con 0x7fd290151840 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd2900cb8b0 0x7fd29006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd2900cb8b0 0x7fd29006b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57784/0 (socket says 172.21.15.8:57784) 2023-10-30T12:49:34.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 -- 172.21.15.8:0/1592278081 learned_addr learned my addr 172.21.15.8:0/1592278081 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:34.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd287fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd290151c20 0x7fd290072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35832/0 (socket says 172.21.15.8:35832) 2023-10-30T12:49:34.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28f7fe640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd290151840 0x7fd2900c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 -- 172.21.15.8:0/1592278081 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd290151840 msgr2=0x7fd2900c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:34.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd290151840 0x7fd2900c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 -- 172.21.15.8:0/1592278081 >> v1:172.21.15.8:6789/0 conn(0x7fd290151c20 legacy=0x7fd290072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28ffff640 1 -- 172.21.15.8:0/1592278081 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290157df0 con 0x7fd2900cb8b0 2023-10-30T12:49:34.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.530+0000 7fd28effd640 1 -- 172.21.15.8:0/1592278081 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826317479 0 0) 0x7fd2900c5030 con 0x7fd290151c20 2023-10-30T12:49:34.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd28ffff640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2900cb8b0 0x7fd29006b600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd288009e50 tx=0x7fd28801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968bf3b02229e6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd28effd640 1 -- 172.21.15.8:0/1592278081 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880af400 con 0x7fd2900cb8b0 2023-10-30T12:49:34.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd28effd640 1 -- 172.21.15.8:0/1592278081 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2880ab050 con 0x7fd2900cb8b0 2023-10-30T12:49:34.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd28effd640 1 -- 172.21.15.8:0/1592278081 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880af780 con 0x7fd2900cb8b0 2023-10-30T12:49:34.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- 172.21.15.8:0/1592278081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2900cb8b0 msgr2=0x7fd29006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2900cb8b0 0x7fd29006b600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd288009e50 tx=0x7fd28801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- 172.21.15.8:0/1592278081 shutdown_connections 2023-10-30T12:49:34.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2900cb8b0 0x7fd29006b600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 --2- 172.21.15.8:0/1592278081 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd290151840 0x7fd2900c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- 172.21.15.8:0/1592278081 >> 172.21.15.8:0/1592278081 conn(0x7fd2900cac70 msgr2=0x7fd2900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:34.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- 172.21.15.8:0/1592278081 shutdown_connections 2023-10-30T12:49:34.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- 172.21.15.8:0/1592278081 wait complete. 2023-10-30T12:49:34.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 Processor -- start 2023-10-30T12:49:34.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.534+0000 7fd295ed6640 1 -- start start 2023-10-30T12:49:34.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd2900cb8b0 0x7fd290071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290151840 0x7fd2900c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290072530 con 0x7fd290151c20 2023-10-30T12:49:34.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290071fe0 con 0x7fd290151840 2023-10-30T12:49:34.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290072250 con 0x7fd2900cb8b0 2023-10-30T12:49:34.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57788/0 (socket says 172.21.15.8:57788) 2023-10-30T12:49:34.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 -- 172.21.15.8:0/450363387 learned_addr learned my addr 172.21.15.8:0/450363387 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:34.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28f7fe640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd2900cb8b0 0x7fd290071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd287fff640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290151840 0x7fd2900c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 -- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd2900cb8b0 msgr2=0x7fd290071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd2900cb8b0 0x7fd290071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 -- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290151840 msgr2=0x7fd2900c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290151840 0x7fd2900c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 -- 172.21.15.8:0/450363387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009b30 con 0x7fd290151c20 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28ffff640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd2880b2040 tx=0x7fd28801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880afd10 con 0x7fd290151c20 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2880afe70 con 0x7fd290151c20 2023-10-30T12:49:34.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2901556c0 con 0x7fd290151c20 2023-10-30T12:49:34.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880ba640 con 0x7fd290151c20 2023-10-30T12:49:34.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd290155c80 con 0x7fd290151c20 2023-10-30T12:49:34.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290066510 con 0x7fd290151c20 2023-10-30T12:49:34.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.538+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd2880b6040 con 0x7fd290151c20 2023-10-30T12:49:34.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.542+0000 7fd28cff9640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 0x7fd270068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:34.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.542+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fd288122190 con 0x7fd290151c20 2023-10-30T12:49:34.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.542+0000 7fd28f7fe640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 0x7fd270068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:34.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.542+0000 7fd28f7fe640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 0x7fd270068990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd280009b60 tx=0x7fd280009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:34.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.546+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd2880ef3d0 con 0x7fd290151c20 2023-10-30T12:49:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.738+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd29004e170 con 0x7fd290151c20 2023-10-30T12:49:34.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.738+0000 7fd28cff9640 1 -- 172.21.15.8:0/450363387 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5729 (secure 0 0 0) 0x7fd288009510 con 0x7fd290151c20 2023-10-30T12:49:34.747 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:34.747 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:32.281718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:resolve","state_seq":27,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:34.747 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 22 2023-10-30T12:49:34.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 msgr2=0x7fd270068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 0x7fd270068990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd280009b60 tx=0x7fd280009190 comp rx=0 tx=0).stop 2023-10-30T12:49:34.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 msgr2=0x7fd2901f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd2880b2040 tx=0x7fd28801bf90 comp rx=0 tx=0).stop 2023-10-30T12:49:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 shutdown_connections 2023-10-30T12:49:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd2700664d0 0x7fd270068990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290151840 0x7fd2900c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd290151c20 0x7fd2901f2320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 --2- 172.21.15.8:0/450363387 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd2900cb8b0 0x7fd290071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 >> 172.21.15.8:0/450363387 conn(0x7fd2900cac70 msgr2=0x7fd2901e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 shutdown_connections 2023-10-30T12:49:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:34.742+0000 7fd295ed6640 1 -- 172.21.15.8:0/450363387 wait complete. 2023-10-30T12:49:34.768 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:36.769 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:36.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 Processor -- start 2023-10-30T12:49:36.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 -- start start 2023-10-30T12:49:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c84151840 0x7f5c840c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c840c5030 con 0x7f5c840cb8b0 2023-10-30T12:49:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c84157830 con 0x7f5c84151c20 2023-10-30T12:49:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8b9d3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c84157b10 con 0x7f5c84151840 2023-10-30T12:49:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c89f70640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f5c840cb8b0 0x7f5c8406b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35840/0 (socket says 172.21.15.8:35840) 2023-10-30T12:49:36.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c89f70640 1 -- 172.21.15.8:0/3304425374 learned_addr learned my addr 172.21.15.8:0/3304425374 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:36.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> v2:172.21.15.46:3300/0 conn(0x7f5c84151840 0x7f5c840c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509166734 0 0) 0x7f5c840c5030 con 0x7f5c840cb8b0 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c70003530 con 0x7f5c840cb8b0 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632844566 0 0) 0x7f5c70003530 con 0x7f5c840cb8b0 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 >> v2:172.21.15.46:3300/0 conn(0x7f5c84151840 msgr2=0x7f5c840c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 --2- 172.21.15.8:0/3304425374 >> v2:172.21.15.46:3300/0 conn(0x7f5c84151840 0x7f5c840c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 >> v1:172.21.15.46:6789/0 conn(0x7f5c84151c20 legacy=0x7f5c84072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84157df0 con 0x7f5c840cb8b0 2023-10-30T12:49:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f5c7c001a00 con 0x7f5c840cb8b0 2023-10-30T12:49:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 >> v1:172.21.15.8:6789/0 conn(0x7f5c840cb8b0 legacy=0x7f5c8406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c700047d0 0x7f5c70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c70005190 0x7f5c7000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c84157b10 con 0x7f5c70003980 2023-10-30T12:49:36.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7000c040 con 0x7f5c70005190 2023-10-30T12:49:36.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7000c200 con 0x7f5c700047d0 2023-10-30T12:49:36.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> v2:172.21.15.46:3300/0 conn(0x7f5c84151840 0x7f5c840c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:36.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c89f70640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c70005190 0x7f5c7000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.918+0000 7f5c88f6e640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c700047d0 0x7f5c70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 -- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c700047d0 msgr2=0x7f5c70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:36.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c700047d0 0x7f5c70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 -- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c70005190 msgr2=0x7f5c7000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:36.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c70005190 0x7f5c7000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 -- 172.21.15.8:0/3304425374 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70003530 con 0x7f5c70003980 2023-10-30T12:49:36.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c89f70640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c70005190 0x7f5c7000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:36.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c89f70640 1 -- 172.21.15.8:0/3304425374 reap_dead start 2023-10-30T12:49:36.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8976f640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c70003da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009c90 tx=0x7f5c8001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b5f69f25cd4ddd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80004020 con 0x7f5c70003980 2023-10-30T12:49:36.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80004180 con 0x7f5c70003980 2023-10-30T12:49:36.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c7bfff640 1 -- 172.21.15.8:0/3304425374 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5890 con 0x7f5c70003980 2023-10-30T12:49:36.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 msgr2=0x7f5c70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c70003da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009c90 tx=0x7f5c8001cbd0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/3304425374 shutdown_connections 2023-10-30T12:49:36.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/3304425374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c70003da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.922+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/3304425374 >> 172.21.15.8:0/3304425374 conn(0x7f5c840cac70 msgr2=0x7f5c840cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:36.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/3304425374 shutdown_connections 2023-10-30T12:49:36.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/3304425374 wait complete. 2023-10-30T12:49:36.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 Processor -- start 2023-10-30T12:49:36.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- start start 2023-10-30T12:49:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c840c94c0 0x7f5c841ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c8406fdf0 0x7f5c84070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c84147f30 con 0x7f5c70003980 2023-10-30T12:49:36.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c841dd800 con 0x7f5c840c94c0 2023-10-30T12:49:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8b9d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c841df360 con 0x7f5c8406fdf0 2023-10-30T12:49:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57810/0 (socket says 172.21.15.8:57810) 2023-10-30T12:49:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 -- 172.21.15.8:0/2365401527 learned_addr learned my addr 172.21.15.8:0/2365401527 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:36.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c88f6e640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c840c94c0 0x7f5c841ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c89f70640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c8406fdf0 0x7f5c84070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 -- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c8406fdf0 msgr2=0x7f5c84070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c8406fdf0 0x7f5c84070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 -- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c840c94c0 msgr2=0x7f5c841ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:36.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c840c94c0 0x7f5c841ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:36.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 -- 172.21.15.8:0/2365401527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c841df640 con 0x7f5c70003980 2023-10-30T12:49:36.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.926+0000 7f5c8976f640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c8001ca00 tx=0x7f5c70005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5a90 con 0x7f5c70003980 2023-10-30T12:49:36.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c800a5bf0 con 0x7f5c70003980 2023-10-30T12:49:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80006040 con 0x7f5c70003980 2023-10-30T12:49:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c841dda50 con 0x7f5c70003980 2023-10-30T12:49:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c841eec70 con 0x7f5c70003980 2023-10-30T12:49:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8404e170 con 0x7f5c70003980 2023-10-30T12:49:36.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c800061a0 con 0x7f5c70003980 2023-10-30T12:49:36.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 0x7f5c68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:36.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f5c8011eb50 con 0x7f5c70003980 2023-10-30T12:49:36.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.930+0000 7f5c88f6e640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 0x7f5c68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:36.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.934+0000 7f5c88f6e640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 0x7f5c68068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c841e1dc0 tx=0x7f5c74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:36.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:36.934+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c800ebd90 con 0x7f5c70003980 2023-10-30T12:49:37.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.130+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c840c41c0 con 0x7f5c70003980 2023-10-30T12:49:37.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.130+0000 7f5c79ffb640 1 -- 172.21.15.8:0/2365401527 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5729 (secure 0 0 0) 0x7f5c800ef3b0 con 0x7f5c70003980 2023-10-30T12:49:37.139 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:37.139 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:32.281718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:resolve","state_seq":27,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 22 2023-10-30T12:49:37.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 msgr2=0x7f5c68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 0x7f5c68068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c841e1dc0 tx=0x7f5c74009040 comp rx=0 tx=0).stop 2023-10-30T12:49:37.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 msgr2=0x7f5c841479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c8001ca00 tx=0x7f5c70005800 comp rx=0 tx=0).stop 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 shutdown_connections 2023-10-30T12:49:37.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5c680664a0 0x7f5c68068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c840c94c0 0x7f5c841ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5c8406fdf0 0x7f5c84070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 --2- 172.21.15.8:0/2365401527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c70003980 0x7f5c841479f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 >> 172.21.15.8:0/2365401527 conn(0x7f5c840cac70 msgr2=0x7f5c840c5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:37.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 shutdown_connections 2023-10-30T12:49:37.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:37.134+0000 7f5c8b9d3640 1 -- 172.21.15.8:0/2365401527 wait complete. 2023-10-30T12:49:37.160 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:39.161 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:39.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 Processor -- start 2023-10-30T12:49:39.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 -- start start 2023-10-30T12:49:39.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd940157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940071e00 con 0x7fd94006c420 2023-10-30T12:49:39.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940070020 con 0x7fd94006b760 2023-10-30T12:49:39.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.306+0000 7fd947f65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940070f10 con 0x7fd94006bb40 2023-10-30T12:49:39.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd946502640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd94006c420 0x7fd9400c57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:58694/0 (socket says 172.21.15.8:58694) 2023-10-30T12:49:39.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd946502640 1 -- 172.21.15.8:0/2995778016 learned_addr learned my addr 172.21.15.8:0/2995778016 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:39.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945500640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd940157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127162569 0 0) 0x7fd940071e00 con 0x7fd94006c420 2023-10-30T12:49:39.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c003530 con 0x7fd94006c420 2023-10-30T12:49:39.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2472064447 0 0) 0x7fd92c003530 con 0x7fd94006c420 2023-10-30T12:49:39.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 msgr2=0x7fd940157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd940157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 >> v1:172.21.15.46:6789/0 conn(0x7fd94006b760 legacy=0x7fd94015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400711f0 con 0x7fd94006c420 2023-10-30T12:49:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fd938001a00 con 0x7fd94006c420 2023-10-30T12:49:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945500640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd940157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:39.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 >> v1:172.21.15.8:6789/0 conn(0x7fd94006c420 legacy=0x7fd9400c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd92c0039b0 0x7fd92c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd92c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c004d10 0x7fd92c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940070f10 con 0x7fd92c0039b0 2023-10-30T12:49:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c00b990 con 0x7fd92c004d10 2023-10-30T12:49:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c00bb00 con 0x7fd94006bb40 2023-10-30T12:49:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd92c0039b0 0x7fd92c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945500640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd92c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd946502640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c004d10 0x7fd92c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 -- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 msgr2=0x7fd92c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd94006bb40 0x7fd92c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 -- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c004d10 msgr2=0x7fd92c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c004d10 0x7fd92c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.310+0000 7fd945d01640 1 -- 172.21.15.8:0/2995778016 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930009420 con 0x7fd92c0039b0 2023-10-30T12:49:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd945d01640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd92c0039b0 0x7fd92c003d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd93c001860 tx=0x7fd93c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61281a521b113d9b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0ab5f0 con 0x7fd92c0039b0 2023-10-30T12:49:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0ab750 con 0x7fd92c0039b0 2023-10-30T12:49:39.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd944cff640 1 -- 172.21.15.8:0/2995778016 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0abb00 con 0x7fd92c0039b0 2023-10-30T12:49:39.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- 172.21.15.8:0/2995778016 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd92c0039b0 msgr2=0x7fd92c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 --2- 172.21.15.8:0/2995778016 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd92c0039b0 0x7fd92c003d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd93c001860 tx=0x7fd93c01c4d0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd946502640 1 -- 172.21.15.8:0/2995778016 reap_dead start 2023-10-30T12:49:39.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- 172.21.15.8:0/2995778016 shutdown_connections 2023-10-30T12:49:39.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- 172.21.15.8:0/2995778016 >> 172.21.15.8:0/2995778016 conn(0x7fd9400cac70 msgr2=0x7fd9400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:39.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- 172.21.15.8:0/2995778016 shutdown_connections 2023-10-30T12:49:39.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- 172.21.15.8:0/2995778016 wait complete. 2023-10-30T12:49:39.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 Processor -- start 2023-10-30T12:49:39.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.314+0000 7fd947f65640 1 -- start start 2023-10-30T12:49:39.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 0x7fd94015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 0x7fd9401f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015b530 con 0x7fd94015b7f0 2023-10-30T12:49:39.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400c9990 con 0x7fd92c0039b0 2023-10-30T12:49:39.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94015abd0 con 0x7fd940157170 2023-10-30T12:49:39.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37926/0 (socket says 172.21.15.8:37926) 2023-10-30T12:49:39.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 -- 172.21.15.8:0/3683450190 learned_addr learned my addr 172.21.15.8:0/3683450190 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:39.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd945500640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 0x7fd9401f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd945d01640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 0x7fd94015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 -- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 msgr2=0x7fd9401f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 0x7fd9401f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 -- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 msgr2=0x7fd94015a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 0x7fd94015a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 -- 172.21.15.8:0/3683450190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400c9be0 con 0x7fd94015b7f0 2023-10-30T12:49:39.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd945500640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 0x7fd9401f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:39.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd945d01640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 0x7fd94015a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:39.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd946502640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd94015b7a0 tx=0x7fd93800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800ec10 con 0x7fd94015b7f0 2023-10-30T12:49:39.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800ed70 con 0x7fd94015b7f0 2023-10-30T12:49:39.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800eee0 con 0x7fd94015b7f0 2023-10-30T12:49:39.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94015ae70 con 0x7fd94015b7f0 2023-10-30T12:49:39.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9400c9ec0 con 0x7fd94015b7f0 2023-10-30T12:49:39.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd940066510 con 0x7fd94015b7f0 2023-10-30T12:49:39.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.318+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd938013640 con 0x7fd94015b7f0 2023-10-30T12:49:39.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.322+0000 7fd9367fc640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 0x7fd924068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:39.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.322+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fd938084480 con 0x7fd94015b7f0 2023-10-30T12:49:39.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.322+0000 7fd945d01640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 0x7fd924068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:39.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.322+0000 7fd945d01640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 0x7fd924068990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0af000 tx=0x7fd93c0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:39.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.326+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd938051790 con 0x7fd94015b7f0 2023-10-30T12:49:39.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.518+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd94004e170 con 0x7fd94015b7f0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.518+0000 7fd9367fc640 1 -- 172.21.15.8:0/3683450190 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fd938054db0 con 0x7fd94015b7f0 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:39.525 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:38.147248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:39.526 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2023-10-30T12:49:39.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 msgr2=0x7fd924068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 0x7fd924068990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0af000 tx=0x7fd93c0a7000 comp rx=0 tx=0).stop 2023-10-30T12:49:39.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 msgr2=0x7fd9401f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:39.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd94015b7a0 tx=0x7fd93800e530 comp rx=0 tx=0).stop 2023-10-30T12:49:39.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 shutdown_connections 2023-10-30T12:49:39.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd9240664d0 0x7fd924068990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd94015b7f0 0x7fd9401f6560 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd940157170 0x7fd9401f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 --2- 172.21.15.8:0/3683450190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92c0039b0 0x7fd94015a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:39.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 >> 172.21.15.8:0/3683450190 conn(0x7fd9400cac70 msgr2=0x7fd940153980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:39.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 shutdown_connections 2023-10-30T12:49:39.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:39.522+0000 7fd947f65640 1 -- 172.21.15.8:0/3683450190 wait complete. 2023-10-30T12:49:39.546 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 23, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 23, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4276": { "addr": "172.21.15.46:6833/3888673612", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 3888673612, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 3888673612, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4276, "incarnation": 9, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 29 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:49:38.147248+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 4276, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:49:41.548 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:41.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 Processor -- start 2023-10-30T12:49:41.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 -- start start 2023-10-30T12:49:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 0x7f056c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 0x7f056c15a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c15dcc0 con 0x7f056c06aeb0 2023-10-30T12:49:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c159800 con 0x7f056c15ac70 2023-10-30T12:49:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057113b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c159d80 con 0x7f056c0c8eb0 2023-10-30T12:49:41.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056bfff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 0x7f056c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 0x7f056c15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057093a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f056c06aeb0 0x7f056c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:52990/0 (socket says 172.21.15.8:52990) 2023-10-30T12:49:41.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f057093a640 1 -- 172.21.15.8:0/2496484777 learned_addr learned my addr 172.21.15.8:0/2496484777 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056bfff640 1 -- 172.21.15.8:0/2496484777 >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 msgr2=0x7f056c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056bfff640 1 --2- 172.21.15.8:0/2496484777 >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 0x7f056c15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056bfff640 1 -- 172.21.15.8:0/2496484777 >> v1:172.21.15.46:6789/0 conn(0x7f056c06aeb0 legacy=0x7f056c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056bfff640 1 -- 172.21.15.8:0/2496484777 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c15a750 con 0x7f056c0c8eb0 2023-10-30T12:49:41.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.698+0000 7f056b7fe640 1 --2- 172.21.15.8:0/2496484777 >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 0x7f056c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:49:41.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f056bfff640 1 --2- 172.21.15.8:0/2496484777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 0x7f056c15cd70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097c0 tx=0x7f055c01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4f9ce4764f16ed server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f056affd640 1 -- 172.21.15.8:0/2496484777 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a8970 con 0x7f056c0c8eb0 2023-10-30T12:49:41.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f056affd640 1 -- 172.21.15.8:0/2496484777 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c0a8ad0 con 0x7f056c0c8eb0 2023-10-30T12:49:41.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f056affd640 1 -- 172.21.15.8:0/2496484777 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a8e60 con 0x7f056c0c8eb0 2023-10-30T12:49:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 -- 172.21.15.8:0/2496484777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 msgr2=0x7f056c15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 --2- 172.21.15.8:0/2496484777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 0x7f056c15cd70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097c0 tx=0x7f055c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:49:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 -- 172.21.15.8:0/2496484777 shutdown_connections 2023-10-30T12:49:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 --2- 172.21.15.8:0/2496484777 >> v2:172.21.15.46:3300/0 conn(0x7f056c15ac70 0x7f056c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 --2- 172.21.15.8:0/2496484777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c0c8eb0 0x7f056c15cd70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 -- 172.21.15.8:0/2496484777 >> 172.21.15.8:0/2496484777 conn(0x7f056c0cef00 msgr2=0x7f056c0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:41.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 -- 172.21.15.8:0/2496484777 shutdown_connections 2023-10-30T12:49:41.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.702+0000 7f057113b640 1 -- 172.21.15.8:0/2496484777 wait complete. 2023-10-30T12:49:41.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 Processor -- start 2023-10-30T12:49:41.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- start start 2023-10-30T12:49:41.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c06aeb0 0x7f056c1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c0c8eb0 0x7f056c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1f4170 con 0x7f056c15ac70 2023-10-30T12:49:41.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1f70b0 con 0x7f056c0c8eb0 2023-10-30T12:49:41.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c158ef0 con 0x7f056c06aeb0 2023-10-30T12:49:41.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37930/0 (socket says 172.21.15.8:37930) 2023-10-30T12:49:41.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 -- 172.21.15.8:0/505169984 learned_addr learned my addr 172.21.15.8:0/505169984 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:41.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f056b7fe640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c0c8eb0 0x7f056c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f056bfff640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c06aeb0 0x7f056c1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 -- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c06aeb0 msgr2=0x7f056c1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c06aeb0 0x7f056c1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 -- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c0c8eb0 msgr2=0x7f056c0c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c0c8eb0 0x7f056c0c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 -- 172.21.15.8:0/505169984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009420 con 0x7f056c15ac70 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057093a640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f056400cab0 tx=0x7f056400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400ac30 con 0x7f056c15ac70 2023-10-30T12:49:41.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056400ad90 con 0x7f056c15ac70 2023-10-30T12:49:41.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564012600 con 0x7f056c15ac70 2023-10-30T12:49:41.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c009420 con 0x7f056c15ac70 2023-10-30T12:49:41.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.706+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1f6780 con 0x7f056c15ac70 2023-10-30T12:49:41.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c04e170 con 0x7f056c15ac70 2023-10-30T12:49:41.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0564012760 con 0x7f056c15ac70 2023-10-30T12:49:41.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f0568ff9640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 0x7f0558068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:41.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f0564083d00 con 0x7f056c15ac70 2023-10-30T12:49:41.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f056bfff640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 0x7f0558068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:41.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.710+0000 7f056bfff640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 0x7f0558068990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b2000 tx=0x7f055c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:41.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.714+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0564050f90 con 0x7f056c15ac70 2023-10-30T12:49:41.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.902+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056c15dcc0 con 0x7f056c15ac70 2023-10-30T12:49:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.902+0000 7f0568ff9640 1 -- 172.21.15.8:0/505169984 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f056c15dcc0 con 0x7f056c15ac70 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:41.911 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:38.147248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 msgr2=0x7f0558068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 0x7f0558068990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b2000 tx=0x7f055c008040 comp rx=0 tx=0).stop 2023-10-30T12:49:41.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 msgr2=0x7f056c1f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:41.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f056400cab0 tx=0x7f056400cf70 comp rx=0 tx=0).stop 2023-10-30T12:49:41.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 shutdown_connections 2023-10-30T12:49:41.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f05580664d0 0x7f0558068990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c0c8eb0 0x7f056c0c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f056c15ac70 0x7f056c1f4680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 --2- 172.21.15.8:0/505169984 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f056c06aeb0 0x7f056c1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 >> 172.21.15.8:0/505169984 conn(0x7f056c0cef00 msgr2=0x7f056c0c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 shutdown_connections 2023-10-30T12:49:41.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:41.906+0000 7f057113b640 1 -- 172.21.15.8:0/505169984 wait complete. 2023-10-30T12:49:41.931 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:43.933 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:44.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 Processor -- start 2023-10-30T12:49:44.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 -- start start 2023-10-30T12:49:44.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd0c0c42a0 0x7fcd0c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c159f20 con 0x7fcd0c153c80 2023-10-30T12:49:44.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c15dec0 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd140c9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c1599a0 con 0x7fcd0c0c42a0 2023-10-30T12:49:44.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37936/0 (socket says 172.21.15.8:37936) 2023-10-30T12:49:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 -- 172.21.15.8:0/722281193 learned_addr learned my addr 172.21.15.8:0/722281193 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd12666640 1 --2- 172.21.15.8:0/722281193 >> v2:172.21.15.46:3300/0 conn(0x7fcd0c0c42a0 0x7fcd0c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 -- 172.21.15.8:0/722281193 >> v2:172.21.15.46:3300/0 conn(0x7fcd0c0c42a0 msgr2=0x7fcd0c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 --2- 172.21.15.8:0/722281193 >> v2:172.21.15.46:3300/0 conn(0x7fcd0c0c42a0 0x7fcd0c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 -- 172.21.15.8:0/722281193 >> v1:172.21.15.46:6789/0 conn(0x7fcd0c153c80 legacy=0x7fcd0c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.082+0000 7fcd11664640 1 -- 172.21.15.8:0/722281193 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c15d620 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd11664640 1 --2- 172.21.15.8:0/722281193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0b1d40 tx=0x7fccfc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58432162dd7eb0c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd10e63640 1 -- 172.21.15.8:0/722281193 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0a89f0 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd10e63640 1 -- 172.21.15.8:0/722281193 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0a8b50 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd10e63640 1 -- 172.21.15.8:0/722281193 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b0440 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 -- 172.21.15.8:0/722281193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 msgr2=0x7fcd0c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 --2- 172.21.15.8:0/722281193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0b1d40 tx=0x7fccfc01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 -- 172.21.15.8:0/722281193 shutdown_connections 2023-10-30T12:49:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 --2- 172.21.15.8:0/722281193 >> v2:172.21.15.46:3300/0 conn(0x7fcd0c0c42a0 0x7fcd0c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 --2- 172.21.15.8:0/722281193 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c15cf90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 -- 172.21.15.8:0/722281193 >> 172.21.15.8:0/722281193 conn(0x7fcd0c0715c0 msgr2=0x7fcd0c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 -- 172.21.15.8:0/722281193 shutdown_connections 2023-10-30T12:49:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.086+0000 7fcd140c9640 1 -- 172.21.15.8:0/722281193 wait complete. 2023-10-30T12:49:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 Processor -- start 2023-10-30T12:49:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- start start 2023-10-30T12:49:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0c42a0 0x7fcd0c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd0c153c80 0x7fcd0c149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c151940 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c14fd40 con 0x7fcd0c0c42a0 2023-10-30T12:49:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c150020 con 0x7fcd0c153c80 2023-10-30T12:49:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37938/0 (socket says 172.21.15.8:37938) 2023-10-30T12:49:44.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 -- 172.21.15.8:0/468362202 learned_addr learned my addr 172.21.15.8:0/468362202 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:44.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd12666640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd0c153c80 0x7fcd0c149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11e65640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0c42a0 0x7fcd0c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 -- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd0c153c80 msgr2=0x7fcd0c149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd0c153c80 0x7fcd0c149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 -- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0c42a0 msgr2=0x7fcd0c150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0c42a0 0x7fcd0c150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 -- 172.21.15.8:0/468362202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc0094d0 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd11664640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0b3040 tx=0x7fccfc01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b0e00 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0bb470 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0bb820 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c1f1f10 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c1f24d0 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.090+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c04e170 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.094+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fccfc0b7070 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.094+0000 7fcd027fc640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 0x7fccf0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:44.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.094+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fccfc122450 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.094+0000 7fcd11e65640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 0x7fccf0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:44.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.094+0000 7fcd11e65640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 0x7fccf0068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000f70 tx=0x7fcd08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:44.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.098+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fccfc0ef630 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.286+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd0c0cad40 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.286+0000 7fcd027fc640 1 -- 172.21.15.8:0/468362202 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5728 (secure 0 0 0) 0x7fccfc0b5090 con 0x7fcd0c0c4a10 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:44.294 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:43.202205+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:44.295 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-10-30T12:49:44.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 msgr2=0x7fccf0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 0x7fccf0068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000f70 tx=0x7fcd08009040 comp rx=0 tx=0).stop 2023-10-30T12:49:44.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 msgr2=0x7fcd0c0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:44.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0b3040 tx=0x7fccfc01ba90 comp rx=0 tx=0).stop 2023-10-30T12:49:44.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 shutdown_connections 2023-10-30T12:49:44.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fccf00664a0 0x7fccf0068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcd0c153c80 0x7fcd0c149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcd0c0c4a10 0x7fcd0c0cd040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 --2- 172.21.15.8:0/468362202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0c42a0 0x7fcd0c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:44.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 >> 172.21.15.8:0/468362202 conn(0x7fcd0c0715c0 msgr2=0x7fcd0c0c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:44.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 shutdown_connections 2023-10-30T12:49:44.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:44.290+0000 7fcd140c9640 1 -- 172.21.15.8:0/468362202 wait complete. 2023-10-30T12:49:44.315 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:44.534 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:44.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:49:46.316 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:46.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 Processor -- start 2023-10-30T12:49:46.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 -- start start 2023-10-30T12:49:46.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7588153c80 0x7f758815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75880c4a10 0x7f75880c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7588159f20 con 0x7f758815a3b0 2023-10-30T12:49:46.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758815dec0 con 0x7f75880c4a10 2023-10-30T12:49:46.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758ee09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75881599a0 con 0x7f7588153c80 2023-10-30T12:49:46.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75880c4a10 0x7f75880c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f75880c4a10 0x7f75880c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37944/0 (socket says 172.21.15.8:37944) 2023-10-30T12:49:46.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 -- 172.21.15.8:0/73144345 learned_addr learned my addr 172.21.15.8:0/73144345 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:46.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f758cba5640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7588153c80 0x7f758815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 -- 172.21.15.8:0/73144345 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7588153c80 msgr2=0x7f758815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:46.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7588153c80 0x7f758815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 -- 172.21.15.8:0/73144345 >> v1:172.21.15.46:6789/0 conn(0x7f758815a3b0 legacy=0x7f75880ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.466+0000 7f757ffff640 1 -- 172.21.15.8:0/73144345 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880c47e0 con 0x7f75880c4a10 2023-10-30T12:49:46.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f757ffff640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75880c4a10 0x7f75880c42a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7578009870 tx=0x7f757801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f05caf9ee8afbbd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f757f7fe640 1 -- 172.21.15.8:0/73144345 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780a8a30 con 0x7f75880c4a10 2023-10-30T12:49:46.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f757f7fe640 1 -- 172.21.15.8:0/73144345 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75780a8b90 con 0x7f75880c4a10 2023-10-30T12:49:46.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f757f7fe640 1 -- 172.21.15.8:0/73144345 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780b0440 con 0x7f75880c4a10 2023-10-30T12:49:46.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 -- 172.21.15.8:0/73144345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75880c4a10 msgr2=0x7f75880c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75880c4a10 0x7f75880c42a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7578009870 tx=0x7f757801ed30 comp rx=0 tx=0).stop 2023-10-30T12:49:46.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 -- 172.21.15.8:0/73144345 shutdown_connections 2023-10-30T12:49:46.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75880c4a10 0x7f75880c42a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 --2- 172.21.15.8:0/73144345 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7588153c80 0x7f758815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 -- 172.21.15.8:0/73144345 >> 172.21.15.8:0/73144345 conn(0x7f75880715c0 msgr2=0x7f75880719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:46.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 -- 172.21.15.8:0/73144345 shutdown_connections 2023-10-30T12:49:46.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.470+0000 7f758ee09640 1 -- 172.21.15.8:0/73144345 wait complete. 2023-10-30T12:49:46.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 Processor -- start 2023-10-30T12:49:46.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 -- start start 2023-10-30T12:49:46.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75880c4a10 0x7f75881f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 0x7f75880c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758815b330 con 0x7f758815a3b0 2023-10-30T12:49:46.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758815c930 con 0x7f7588153c80 2023-10-30T12:49:46.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75881f31b0 con 0x7f75880c4a10 2023-10-30T12:49:46.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37956/0 (socket says 172.21.15.8:37956) 2023-10-30T12:49:46.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 -- 172.21.15.8:0/3896429649 learned_addr learned my addr 172.21.15.8:0/3896429649 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:46.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758cba5640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75880c4a10 0x7f75881f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f757ffff640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 0x7f75880c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 -- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75880c4a10 msgr2=0x7f75881f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75880c4a10 0x7f75881f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 -- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 msgr2=0x7f75880c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 0x7f75880c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 -- 172.21.15.8:0/3896429649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780094d0 con 0x7f758815a3b0 2023-10-30T12:49:46.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f757ffff640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 0x7f75880c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:46.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758d3a6640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f758000c760 tx=0x7f758000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580003b80 con 0x7f758815a3b0 2023-10-30T12:49:46.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580003ce0 con 0x7f758815a3b0 2023-10-30T12:49:46.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75880b1ae0 con 0x7f758815a3b0 2023-10-30T12:49:46.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.474+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000aa30 con 0x7f758815a3b0 2023-10-30T12:49:46.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758815c000 con 0x7f758815a3b0 2023-10-30T12:49:46.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758804e170 con 0x7f758815a3b0 2023-10-30T12:49:46.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f758000ab90 con 0x7f758815a3b0 2023-10-30T12:49:46.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f757d7fa640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 0x7f7574068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:46.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f7580083db0 con 0x7f758815a3b0 2023-10-30T12:49:46.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f758cba5640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 0x7f7574068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:46.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.478+0000 7f758cba5640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 0x7f7574068990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7584007a60 tx=0x7f7584008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:46.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.482+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7580088070 con 0x7f758815a3b0 2023-10-30T12:49:46.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.670+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f758815ca70 con 0x7f758815a3b0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.670+0000 7f757d7fa640 1 -- 172.21.15.8:0/3896429649 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5728 (secure 0 0 0) 0x7f7580051020 con 0x7f758815a3b0 2023-10-30T12:49:46.679 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:46.680 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:43.202205+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:46.682 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-10-30T12:49:46.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 msgr2=0x7f7574068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 0x7f7574068990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7584007a60 tx=0x7f7584008040 comp rx=0 tx=0).stop 2023-10-30T12:49:46.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 msgr2=0x7f758814eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:46.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f758000c760 tx=0x7f758000cc20 comp rx=0 tx=0).stop 2023-10-30T12:49:46.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 shutdown_connections 2023-10-30T12:49:46.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f75740664d0 0x7f7574068990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7588153c80 0x7f75880c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f758815a3b0 0x7f758814eab0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 --2- 172.21.15.8:0/3896429649 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f75880c4a10 0x7f75881f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:46.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 >> 172.21.15.8:0/3896429649 conn(0x7f75880715c0 msgr2=0x7f75881586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:46.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 shutdown_connections 2023-10-30T12:49:46.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:46.674+0000 7f758ee09640 1 -- 172.21.15.8:0/3896429649 wait complete. 2023-10-30T12:49:46.699 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:48.700 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:48.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 Processor -- start 2023-10-30T12:49:48.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 -- start start 2023-10-30T12:49:48.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc1f80c4a10 0x7fc1f815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 0x7fc1f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f8159f20 con 0x7fc1f8153c80 2023-10-30T12:49:48.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f815dec0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1ffb65640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f81599a0 con 0x7fc1f80c42a0 2023-10-30T12:49:48.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd100640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc1f80c4a10 0x7fc1f815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd901640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fc1f8153c80 0x7fc1f815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:58698/0 (socket says 172.21.15.8:58698) 2023-10-30T12:49:48.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd901640 1 -- 172.21.15.8:0/3857784991 learned_addr learned my addr 172.21.15.8:0/3857784991 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:48.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fe102640 1 --2- 172.21.15.8:0/3857784991 >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 0x7fc1f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fc8ff640 1 -- 172.21.15.8:0/3857784991 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143317392 0 0) 0x7fc1f8159f20 con 0x7fc1f8153c80 2023-10-30T12:49:48.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fc8ff640 1 -- 172.21.15.8:0/3857784991 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e4003530 con 0x7fc1f8153c80 2023-10-30T12:49:48.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd100640 1 -- 172.21.15.8:0/3857784991 >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 msgr2=0x7fc1f80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:48.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd100640 1 --2- 172.21.15.8:0/3857784991 >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 0x7fc1f80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:48.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd100640 1 -- 172.21.15.8:0/3857784991 >> v1:172.21.15.8:6789/0 conn(0x7fc1f8153c80 legacy=0x7fc1f815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:48.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fd100640 1 -- 172.21.15.8:0/3857784991 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f815d620 con 0x7fc1f80c4a10 2023-10-30T12:49:48.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.850+0000 7fc1fe102640 1 --2- 172.21.15.8:0/3857784991 >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 0x7fc1f80ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:48.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1fd100640 1 --2- 172.21.15.8:0/3857784991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f815cf90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b1d40 tx=0x7fc1e801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d6c8d176dc9f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:48.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1fc8ff640 1 -- 172.21.15.8:0/3857784991 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80a8950 con 0x7fc1f80c4a10 2023-10-30T12:49:48.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1fc8ff640 1 -- 172.21.15.8:0/3857784991 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e80a8ab0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1fc8ff640 1 -- 172.21.15.8:0/3857784991 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80a8e40 con 0x7fc1f80c4a10 2023-10-30T12:49:48.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3857784991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 msgr2=0x7fc1f815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:48.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3857784991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f815cf90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b1d40 tx=0x7fc1e801ec50 comp rx=0 tx=0).stop 2023-10-30T12:49:48.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3857784991 shutdown_connections 2023-10-30T12:49:48.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3857784991 >> v2:172.21.15.46:3300/0 conn(0x7fc1f80c42a0 0x7fc1f80ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:48.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3857784991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f815cf90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:48.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3857784991 >> 172.21.15.8:0/3857784991 conn(0x7fc1f80715c0 msgr2=0x7fc1f80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3857784991 shutdown_connections 2023-10-30T12:49:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.854+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3857784991 wait complete. 2023-10-30T12:49:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 Processor -- start 2023-10-30T12:49:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- start start 2023-10-30T12:49:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc1f80c42a0 0x7fc1f8154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f8153c80 0x7fc1f8155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8154840 con 0x7fc1f80c4a10 2023-10-30T12:49:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81569a0 con 0x7fc1f8153c80 2023-10-30T12:49:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8154ff0 con 0x7fc1f80c42a0 2023-10-30T12:49:48.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37974/0 (socket says 172.21.15.8:37974) 2023-10-30T12:49:48.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 -- 172.21.15.8:0/3853618839 learned_addr learned my addr 172.21.15.8:0/3853618839 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:48.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd901640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc1f80c42a0 0x7fc1f8154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fe102640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f8153c80 0x7fc1f8155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 -- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc1f80c42a0 msgr2=0x7fc1f8154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:48.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc1f80c42a0 0x7fc1f8154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:48.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 -- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f8153c80 msgr2=0x7fc1f8155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:48.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f8153c80 0x7fc1f8155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:48.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 -- 172.21.15.8:0/3853618839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8009420 con 0x7fc1f80c4a10 2023-10-30T12:49:48.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1fd100640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1e801b860 tx=0x7fc1e801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:48.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80b0df0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e80b9410 con 0x7fc1f80c4a10 2023-10-30T12:49:48.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80b97a0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f8156bd0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.858+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f815c690 con 0x7fc1f80c4a10 2023-10-30T12:49:48.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f8159b90 con 0x7fc1f80c4a10 2023-10-30T12:49:48.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc1e80b7040 con 0x7fc1f80c4a10 2023-10-30T12:49:48.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1ee7fc640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 0x7fc1dc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:48.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fc1e8121bf0 con 0x7fc1f80c4a10 2023-10-30T12:49:48.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1fd901640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 0x7fc1dc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:48.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.862+0000 7fc1fd901640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 0x7fc1dc068990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40011f0 tx=0x7fc1f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:48.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:48.866+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1f8159b90 con 0x7fc1f80c4a10 2023-10-30T12:49:49.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1f8159b90 con 0x7fc1f80c4a10 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ee7fc640 1 -- 172.21.15.8:0/3853618839 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5728 (secure 0 0 0) 0x7fc1f8159b90 con 0x7fc1f80c4a10 2023-10-30T12:49:49.064 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:43.202205+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:49.065 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-10-30T12:49:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 msgr2=0x7fc1dc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 0x7fc1dc068990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40011f0 tx=0x7fc1f4009040 comp rx=0 tx=0).stop 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 msgr2=0x7fc1f80c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1e801b860 tx=0x7fc1e801b9f0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 shutdown_connections 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc1dc0664d0 0x7fc1dc068990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f8153c80 0x7fc1f8155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1f80c4a10 0x7fc1f80c89d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 --2- 172.21.15.8:0/3853618839 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc1f80c42a0 0x7fc1f8154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 >> 172.21.15.8:0/3853618839 conn(0x7fc1f80715c0 msgr2=0x7fc1f8158480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 shutdown_connections 2023-10-30T12:49:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:49.058+0000 7fc1ffb65640 1 -- 172.21.15.8:0/3853618839 wait complete. 2023-10-30T12:49:49.086 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:49:51.087 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:51.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 Processor -- start 2023-10-30T12:49:51.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 -- start start 2023-10-30T12:49:51.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6900153c80 0x7f690015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f69000c4a10 0x7f69000c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6900159f20 con 0x7f690015a3b0 2023-10-30T12:49:51.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690015dec0 con 0x7f69000c4a10 2023-10-30T12:49:51.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f69051b8640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69001599a0 con 0x7f6900153c80 2023-10-30T12:49:51.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f69000c4a10 0x7f69000c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68ff59e640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f690015a3b0 0x7f69000ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40832/0 (socket says 172.21.15.8:40832) 2023-10-30T12:49:51.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68ff59e640 1 -- 172.21.15.8:0/2450857261 learned_addr learned my addr 172.21.15.8:0/2450857261 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:51.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f69000c4a10 0x7f69000c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35078/0 (socket says 172.21.15.8:35078) 2023-10-30T12:49:51.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 -- 172.21.15.8:0/2450857261 >> v2:172.21.15.46:3300/0 conn(0x7f6900153c80 msgr2=0x7f690015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:51.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 --2- 172.21.15.8:0/2450857261 >> v2:172.21.15.46:3300/0 conn(0x7f6900153c80 0x7f690015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 -- 172.21.15.8:0/2450857261 >> v1:172.21.15.8:6789/0 conn(0x7f690015a3b0 legacy=0x7f69000ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68f7fff640 1 -- 172.21.15.8:0/2450857261 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000c47e0 con 0x7f69000c4a10 2023-10-30T12:49:51.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.238+0000 7f68fe59c640 1 -- 172.21.15.8:0/2450857261 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829786573 0 0) 0x7f6900159f20 con 0x7f690015a3b0 2023-10-30T12:49:51.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f68f7fff640 1 --2- 172.21.15.8:0/2450857261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69000c4a10 0x7f69000c42a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69000d00f0 tx=0x7f68e801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e682b4f186d0ba server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f68fe59c640 1 -- 172.21.15.8:0/2450857261 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80a8950 con 0x7f69000c4a10 2023-10-30T12:49:51.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f68fe59c640 1 -- 172.21.15.8:0/2450857261 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80a8ab0 con 0x7f69000c4a10 2023-10-30T12:49:51.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f68fe59c640 1 -- 172.21.15.8:0/2450857261 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80a8e40 con 0x7f69000c4a10 2023-10-30T12:49:51.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 -- 172.21.15.8:0/2450857261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69000c4a10 msgr2=0x7f69000c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 --2- 172.21.15.8:0/2450857261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69000c4a10 0x7f69000c42a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69000d00f0 tx=0x7f68e801ec50 comp rx=0 tx=0).stop 2023-10-30T12:49:51.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 -- 172.21.15.8:0/2450857261 shutdown_connections 2023-10-30T12:49:51.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 --2- 172.21.15.8:0/2450857261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69000c4a10 0x7f69000c42a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 --2- 172.21.15.8:0/2450857261 >> v2:172.21.15.46:3300/0 conn(0x7f6900153c80 0x7f690015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 -- 172.21.15.8:0/2450857261 >> 172.21.15.8:0/2450857261 conn(0x7f69000715c0 msgr2=0x7f69000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:51.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 -- 172.21.15.8:0/2450857261 shutdown_connections 2023-10-30T12:49:51.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.242+0000 7f69051b8640 1 -- 172.21.15.8:0/2450857261 wait complete. 2023-10-30T12:49:51.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 Processor -- start 2023-10-30T12:49:51.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- start start 2023-10-30T12:49:51.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 0x7f690014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6900153c80 0x7f6900150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690014df30 con 0x7f690015a3b0 2023-10-30T12:49:51.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6900155710 con 0x7f69000c4a10 2023-10-30T12:49:51.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69001f2310 con 0x7f6900153c80 2023-10-30T12:49:51.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35084/0 (socket says 172.21.15.8:35084) 2023-10-30T12:49:51.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 -- 172.21.15.8:0/134656656 learned_addr learned my addr 172.21.15.8:0/134656656 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:51.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68fed9d640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 0x7f690014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68f7fff640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6900153c80 0x7f6900150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 -- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6900153c80 msgr2=0x7f6900150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6900153c80 0x7f6900150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 -- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 msgr2=0x7f690014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 0x7f690014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 -- 172.21.15.8:0/134656656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8009420 con 0x7f690015a3b0 2023-10-30T12:49:51.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68fed9d640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 0x7f690014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:51.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68ff59e640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c780 tx=0x7f68f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800ce90 con 0x7f690015a3b0 2023-10-30T12:49:51.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800a5e0 con 0x7f690015a3b0 2023-10-30T12:49:51.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800a970 con 0x7f690015a3b0 2023-10-30T12:49:51.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69001f25f0 con 0x7f690015a3b0 2023-10-30T12:49:51.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.246+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69001f2bb0 con 0x7f690015a3b0 2023-10-30T12:49:51.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690004e170 con 0x7f690015a3b0 2023-10-30T12:49:51.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68f800aad0 con 0x7f690015a3b0 2023-10-30T12:49:51.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f68f77fe640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 0x7f68e0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:51.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f68f80833e0 con 0x7f690015a3b0 2023-10-30T12:49:51.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f68fed9d640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 0x7f68e0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:51.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.250+0000 7f68fed9d640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 0x7f68e0068990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:51.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.254+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68f80506f0 con 0x7f690015a3b0 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6900157a60 con 0x7f690015a3b0 2023-10-30T12:49:51.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f68f77fe640 1 -- 172.21.15.8:0/134656656 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5728 (secure 0 0 0) 0x7f68f8053d10 con 0x7f690015a3b0 2023-10-30T12:49:51.452 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:51.452 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:48.267086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:51.453 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2023-10-30T12:49:51.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 msgr2=0x7f68e0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 0x7f68e0068990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).stop 2023-10-30T12:49:51.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 msgr2=0x7f6900149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:51.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c780 tx=0x7f68f800cc40 comp rx=0 tx=0).stop 2023-10-30T12:49:51.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 shutdown_connections 2023-10-30T12:49:51.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68e00664d0 0x7f68e0068990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f690015a3b0 0x7f6900149aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6900153c80 0x7f6900150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 --2- 172.21.15.8:0/134656656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69000c4a10 0x7f690014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 >> 172.21.15.8:0/134656656 conn(0x7f69000715c0 msgr2=0x7f69000c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 shutdown_connections 2023-10-30T12:49:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:51.446+0000 7f69051b8640 1 -- 172.21.15.8:0/134656656 wait complete. 2023-10-30T12:49:51.473 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=4244 2023-10-30T12:49:51.473 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.e 2023-10-30T12:49:52.413 INFO:tasks.ceph.mds.a.smithi008.stderr:2023-10-30T12:49:52.406+0000 7fe0693b9640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:49:54.474 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-30T12:49:54.474 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:49:54.475 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:49:54.479 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:49:54.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:54.568 INFO:tasks.ceph.mds.e.smithi008.stdout:starting mds.e at 2023-10-30T12:49:54.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.618+0000 7fdf45a6b640 1 Processor -- start 2023-10-30T12:49:54.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.618+0000 7fdf45a6b640 1 -- start start 2023-10-30T12:49:54.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf45a6b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf40070520 0x7fdf40070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:54.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf45a6b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf400c2310 con 0x7fdf40150830 2023-10-30T12:49:54.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf45a6b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf40156710 con 0x7fdf400cb7d0 2023-10-30T12:49:54.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf45a6b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4006bc10 con 0x7fdf40070520 2023-10-30T12:49:54.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3f7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fdf40150830 0x7fdf4006b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40840/0 (socket says 172.21.15.8:40840) 2023-10-30T12:49:54.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3f7fe640 1 -- 172.21.15.8:0/2107780012 learned_addr learned my addr 172.21.15.8:0/2107780012 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:49:54.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3ffff640 1 --2- 172.21.15.8:0/2107780012 >> v2:172.21.15.8:3300/0 conn(0x7fdf40070520 0x7fdf40070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:54.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3ffff640 1 -- 172.21.15.8:0/2107780012 >> v1:172.21.15.46:6789/0 conn(0x7fdf400cb7d0 legacy=0x7fdf400c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3ffff640 1 -- 172.21.15.8:0/2107780012 >> v1:172.21.15.8:6789/0 conn(0x7fdf40150830 legacy=0x7fdf4006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3ffff640 1 -- 172.21.15.8:0/2107780012 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400c4360 con 0x7fdf40070520 2023-10-30T12:49:54.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3e7fc640 1 -- 172.21.15.8:0/2107780012 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810440969 0 0) 0x7fdf400c2310 con 0x7fdf40150830 2023-10-30T12:49:54.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.622+0000 7fdf3ffff640 1 --2- 172.21.15.8:0/2107780012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40070520 0x7fdf40070990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009d90 tx=0x7fdf3801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42feccc1c099cfc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:54.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3e7fc640 1 -- 172.21.15.8:0/2107780012 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380af600 con 0x7fdf40070520 2023-10-30T12:49:54.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3e7fc640 1 -- 172.21.15.8:0/2107780012 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf380ab050 con 0x7fdf40070520 2023-10-30T12:49:54.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3e7fc640 1 -- 172.21.15.8:0/2107780012 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380af980 con 0x7fdf40070520 2023-10-30T12:49:54.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- 172.21.15.8:0/2107780012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40070520 msgr2=0x7fdf40070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/2107780012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40070520 0x7fdf40070990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009d90 tx=0x7fdf3801f130 comp rx=0 tx=0).stop 2023-10-30T12:49:54.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- 172.21.15.8:0/2107780012 shutdown_connections 2023-10-30T12:49:54.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/2107780012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40070520 0x7fdf40070990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- 172.21.15.8:0/2107780012 >> 172.21.15.8:0/2107780012 conn(0x7fdf400cac70 msgr2=0x7fdf400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:54.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- 172.21.15.8:0/2107780012 shutdown_connections 2023-10-30T12:49:54.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- 172.21.15.8:0/2107780012 wait complete. 2023-10-30T12:49:54.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 Processor -- start 2023-10-30T12:49:54.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- start start 2023-10-30T12:49:54.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf40070520 0x7fdf4006bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:54.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 0x7fdf401577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:54.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:54.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40158880 con 0x7fdf40150830 2023-10-30T12:49:54.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40149a10 con 0x7fdf400cb7d0 2023-10-30T12:49:54.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf45a6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40149cf0 con 0x7fdf40070520 2023-10-30T12:49:54.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:54.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35130/0 (socket says 172.21.15.8:35130) 2023-10-30T12:49:54.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3ffff640 1 -- 172.21.15.8:0/1609391183 learned_addr learned my addr 172.21.15.8:0/1609391183 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:54.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.626+0000 7fdf3f7fe640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf40070520 0x7fdf4006bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:54.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3effd640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 0x7fdf401577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:54.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 -- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf40070520 msgr2=0x7fdf4006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf40070520 0x7fdf4006bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 -- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 msgr2=0x7fdf401577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 0x7fdf401577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 -- 172.21.15.8:0/1609391183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009a70 con 0x7fdf40150830 2023-10-30T12:49:54.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3effd640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 0x7fdf401577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:49:54.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf3ffff640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdf3801cd60 tx=0x7fdf3801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:54.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380b9610 con 0x7fdf40150830 2023-10-30T12:49:54.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf380b9770 con 0x7fdf40150830 2023-10-30T12:49:54.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf40159710 con 0x7fdf40150830 2023-10-30T12:49:54.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380b9b00 con 0x7fdf40150830 2023-10-30T12:49:54.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf40159cd0 con 0x7fdf40150830 2023-10-30T12:49:54.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf40066510 con 0x7fdf40150830 2023-10-30T12:49:54.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf380b6050 con 0x7fdf40150830 2023-10-30T12:49:54.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 0x7fdf2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:54.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.630+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdf38122620 con 0x7fdf40150830 2023-10-30T12:49:54.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.634+0000 7fdf3f7fe640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 0x7fdf2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:54.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.634+0000 7fdf3f7fe640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 0x7fdf2c068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007c40 tx=0x7fdf30007b90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:54.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.634+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf380ef950 con 0x7fdf40150830 2023-10-30T12:49:54.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf4004e170 con 0x7fdf40150830 2023-10-30T12:49:54.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf27fff640 1 -- 172.21.15.8:0/1609391183 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5728 (secure 0 0 0) 0x7fdf380f2f70 con 0x7fdf40150830 2023-10-30T12:49:54.833 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:54.833 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:48.267086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:54.833 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2023-10-30T12:49:54.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 msgr2=0x7fdf2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 0x7fdf2c068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007c40 tx=0x7fdf30007b90 comp rx=0 tx=0).stop 2023-10-30T12:49:54.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 msgr2=0x7fdf40147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:54.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdf3801cd60 tx=0x7fdf3801bed0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 shutdown_connections 2023-10-30T12:49:54.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf2c0664a0 0x7fdf2c068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf400cb7d0 0x7fdf401577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf40150830 0x7fdf40147540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 --2- 172.21.15.8:0/1609391183 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf40070520 0x7fdf4006bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:54.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.826+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 >> 172.21.15.8:0/1609391183 conn(0x7fdf400cac70 msgr2=0x7fdf400c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:54.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.830+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 shutdown_connections 2023-10-30T12:49:54.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:54.830+0000 7fdf45a6b640 1 -- 172.21.15.8:0/1609391183 wait complete. 2023-10-30T12:49:54.854 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-30T12:49:56.855 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:49:57.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.994+0000 7f68aadf8640 1 Processor -- start 2023-10-30T12:49:57.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.994+0000 7f68aadf8640 1 -- start start 2023-10-30T12:49:57.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68aadf8640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5ba0 0x7f68a40c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68aadf8640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f68a40c5f80 0x7f68a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68aadf8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a415cda0 con 0x7f68a40c9b40 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68aadf8640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a415d990 con 0x7f68a40c5f80 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68aadf8640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4153f60 con 0x7f68a40c5ba0 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f68a40c5f80 0x7f68a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f68a40c5f80 0x7f68a4069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35136/0 (socket says 172.21.15.8:35136) 2023-10-30T12:49:57.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 -- 172.21.15.8:0/633176629 learned_addr learned my addr 172.21.15.8:0/633176629 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:57.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a8b94640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5ba0 0x7f68a40c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 -- 172.21.15.8:0/633176629 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5ba0 msgr2=0x7f68a40c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:49:57.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5ba0 0x7f68a40c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 -- 172.21.15.8:0/633176629 >> v1:172.21.15.46:6789/0 conn(0x7f68a40c9b40 legacy=0x7f68a41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:56.998+0000 7f68a3fff640 1 -- 172.21.15.8:0/633176629 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a406f490 con 0x7f68a40c5f80 2023-10-30T12:49:57.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68a3fff640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c5f80 0x7f68a4069e90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68900098a0 tx=0x7f689001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eaa3ce8fa6616d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:57.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68a37fe640 1 -- 172.21.15.8:0/633176629 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68900ab600 con 0x7f68a40c5f80 2023-10-30T12:49:57.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68a37fe640 1 -- 172.21.15.8:0/633176629 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68900a7050 con 0x7f68a40c5f80 2023-10-30T12:49:57.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68a37fe640 1 -- 172.21.15.8:0/633176629 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68900ab980 con 0x7f68a40c5f80 2023-10-30T12:49:57.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- 172.21.15.8:0/633176629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c5f80 msgr2=0x7f68a4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c5f80 0x7f68a4069e90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68900098a0 tx=0x7f689001c520 comp rx=0 tx=0).stop 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- 172.21.15.8:0/633176629 shutdown_connections 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c5f80 0x7f68a4069e90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 --2- 172.21.15.8:0/633176629 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5ba0 0x7f68a40c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- 172.21.15.8:0/633176629 >> 172.21.15.8:0/633176629 conn(0x7f68a406afa0 msgr2=0x7f68a406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- 172.21.15.8:0/633176629 shutdown_connections 2023-10-30T12:49:57.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- 172.21.15.8:0/633176629 wait complete. 2023-10-30T12:49:57.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 Processor -- start 2023-10-30T12:49:57.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 -- start start 2023-10-30T12:49:57.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.002+0000 7f68aadf8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a40c5ba0 0x7f68a40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5f80 0x7f68a41523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a415bf80 con 0x7f68a40c9b40 2023-10-30T12:49:57.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4151710 con 0x7f68a40c5ba0 2023-10-30T12:49:57.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a415c1f0 con 0x7f68a40c5f80 2023-10-30T12:49:57.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35144/0 (socket says 172.21.15.8:35144) 2023-10-30T12:49:57.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 -- 172.21.15.8:0/4066287688 learned_addr learned my addr 172.21.15.8:0/4066287688 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:49:57.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a8b94640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a40c5ba0 0x7f68a40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a3fff640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5f80 0x7f68a41523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 -- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5f80 msgr2=0x7f68a41523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5f80 0x7f68a41523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 -- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a40c5ba0 msgr2=0x7f68a40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a40c5ba0 0x7f68a40c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 -- 172.21.15.8:0/4066287688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900094d0 con 0x7f68a40c9b40 2023-10-30T12:49:57.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a9395640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c730 tx=0x7f689c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:57.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c003b80 con 0x7f68a40c9b40 2023-10-30T12:49:57.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c003ce0 con 0x7f68a40c9b40 2023-10-30T12:49:57.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a415c4d0 con 0x7f68a40c9b40 2023-10-30T12:49:57.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c005310 con 0x7f68a40c9b40 2023-10-30T12:49:57.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a415ca90 con 0x7f68a40c9b40 2023-10-30T12:49:57.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a404e170 con 0x7f68a40c9b40 2023-10-30T12:49:57.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.006+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f689c005680 con 0x7f68a40c9b40 2023-10-30T12:49:57.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.010+0000 7f68a17fa640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 0x7f6888068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:49:57.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.010+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f689c083ee0 con 0x7f68a40c9b40 2023-10-30T12:49:57.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.010+0000 7f68a8b94640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 0x7f6888068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:49:57.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.010+0000 7f68a8b94640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 0x7f6888068990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6898009b60 tx=0x7f6898009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:49:57.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.014+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f689c051170 con 0x7f68a40c9b40 2023-10-30T12:49:57.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68a406b520 con 0x7f68a40c9b40 2023-10-30T12:49:57.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68a17fa640 1 -- 172.21.15.8:0/4066287688 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f689c054790 con 0x7f68a40c9b40 2023-10-30T12:49:57.209 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:49:57.209 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:49:48.267086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:49:57.209 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 26 2023-10-30T12:49:57.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 msgr2=0x7f6888068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 0x7f6888068990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6898009b60 tx=0x7f6898009190 comp rx=0 tx=0).stop 2023-10-30T12:49:57.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 msgr2=0x7f68a414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:49:57.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c730 tx=0x7f689c00cbf0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 shutdown_connections 2023-10-30T12:49:57.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f68880664d0 0x7f6888068990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f68a40c9b40 0x7f68a414afd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f68a40c5f80 0x7f68a41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 --2- 172.21.15.8:0/4066287688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a40c5ba0 0x7f68a40c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:49:57.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.202+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 >> 172.21.15.8:0/4066287688 conn(0x7f68a406afa0 msgr2=0x7f68a4071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:49:57.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.206+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 shutdown_connections 2023-10-30T12:49:57.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:49:57.206+0000 7f68aadf8640 1 -- 172.21.15.8:0/4066287688 wait complete. 2023-10-30T12:49:57.229 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-30T12:49:57.230 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 40 secs before thrashing 2023-10-30T12:49:57.504 INFO:tasks.workunit.client.0.smithi008.stdout:Delete all the files and directories ... 2023-10-30T12:50:02.854 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/19' 2023-10-30T12:50:02.855 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/15' 2023-10-30T12:50:02.855 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/31' 2023-10-30T12:50:02.855 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/36' 2023-10-30T12:50:02.855 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/20' 2023-10-30T12:50:02.856 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/12' 2023-10-30T12:50:02.856 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/39' 2023-10-30T12:50:02.856 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/10' 2023-10-30T12:50:02.856 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/34' 2023-10-30T12:50:02.856 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/38' 2023-10-30T12:50:02.857 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/11' 2023-10-30T12:50:02.857 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/16' 2023-10-30T12:50:02.857 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/32' 2023-10-30T12:50:02.857 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/30' 2023-10-30T12:50:02.857 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/14' 2023-10-30T12:50:02.858 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/18' 2023-10-30T12:50:02.858 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/13' 2023-10-30T12:50:02.858 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/33' 2023-10-30T12:50:02.858 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/17' 2023-10-30T12:50:02.858 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/35' 2023-10-30T12:50:02.859 INFO:tasks.workunit.client.0.smithi008.stdout:removed './100/37' 2023-10-30T12:50:02.859 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './100' 2023-10-30T12:50:02.859 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/19' 2023-10-30T12:50:02.859 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/15' 2023-10-30T12:50:02.859 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/31' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/36' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/20' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/12' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/39' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/10' 2023-10-30T12:50:02.860 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/34' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/38' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/11' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/16' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/32' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/30' 2023-10-30T12:50:02.861 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/14' 2023-10-30T12:50:02.862 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/18' 2023-10-30T12:50:02.862 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/13' 2023-10-30T12:50:02.862 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/33' 2023-10-30T12:50:02.862 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/17' 2023-10-30T12:50:02.862 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/35' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed './101/37' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './101' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/19' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/15' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/31' 2023-10-30T12:50:02.863 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/36' 2023-10-30T12:50:02.864 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/20' 2023-10-30T12:50:02.864 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/12' 2023-10-30T12:50:02.864 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/39' 2023-10-30T12:50:02.864 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/10' 2023-10-30T12:50:02.864 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/34' 2023-10-30T12:50:02.865 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/38' 2023-10-30T12:50:02.865 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/11' 2023-10-30T12:50:02.865 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/16' 2023-10-30T12:50:02.865 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/32' 2023-10-30T12:50:02.865 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/30' 2023-10-30T12:50:02.866 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/14' 2023-10-30T12:50:02.866 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/18' 2023-10-30T12:50:02.866 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/13' 2023-10-30T12:50:02.866 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/33' 2023-10-30T12:50:02.866 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/17' 2023-10-30T12:50:02.867 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/35' 2023-10-30T12:50:02.867 INFO:tasks.workunit.client.0.smithi008.stdout:removed './102/37' 2023-10-30T12:50:02.867 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './102' 2023-10-30T12:50:02.867 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/19' 2023-10-30T12:50:02.867 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/15' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/31' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/36' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/20' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/12' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/39' 2023-10-30T12:50:02.868 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/10' 2023-10-30T12:50:02.869 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/34' 2023-10-30T12:50:02.869 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/38' 2023-10-30T12:50:02.869 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/11' 2023-10-30T12:50:02.869 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/16' 2023-10-30T12:50:02.869 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/32' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/30' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/14' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/18' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/13' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/33' 2023-10-30T12:50:02.870 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/17' 2023-10-30T12:50:02.871 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/35' 2023-10-30T12:50:02.871 INFO:tasks.workunit.client.0.smithi008.stdout:removed './103/37' 2023-10-30T12:50:02.871 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './103' 2023-10-30T12:50:02.871 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/19' 2023-10-30T12:50:02.871 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/15' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/31' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/36' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/20' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/12' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/39' 2023-10-30T12:50:02.872 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/10' 2023-10-30T12:50:02.873 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/34' 2023-10-30T12:50:02.873 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/38' 2023-10-30T12:50:02.873 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/11' 2023-10-30T12:50:02.873 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/16' 2023-10-30T12:50:02.873 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/32' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/30' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/14' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/18' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/13' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/33' 2023-10-30T12:50:02.874 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/17' 2023-10-30T12:50:02.875 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/35' 2023-10-30T12:50:02.875 INFO:tasks.workunit.client.0.smithi008.stdout:removed './104/37' 2023-10-30T12:50:02.875 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './104' 2023-10-30T12:50:02.875 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/19' 2023-10-30T12:50:02.875 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/15' 2023-10-30T12:50:02.876 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/31' 2023-10-30T12:50:02.876 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/36' 2023-10-30T12:50:02.876 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/20' 2023-10-30T12:50:02.876 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/12' 2023-10-30T12:50:02.877 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/39' 2023-10-30T12:50:02.877 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/10' 2023-10-30T12:50:02.877 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/34' 2023-10-30T12:50:02.877 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/38' 2023-10-30T12:50:02.877 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/11' 2023-10-30T12:50:02.878 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/16' 2023-10-30T12:50:02.878 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/32' 2023-10-30T12:50:02.878 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/30' 2023-10-30T12:50:02.878 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/14' 2023-10-30T12:50:02.878 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/18' 2023-10-30T12:50:02.879 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/13' 2023-10-30T12:50:02.879 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/33' 2023-10-30T12:50:02.879 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/17' 2023-10-30T12:50:02.879 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/35' 2023-10-30T12:50:02.880 INFO:tasks.workunit.client.0.smithi008.stdout:removed './105/37' 2023-10-30T12:50:02.880 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './105' 2023-10-30T12:50:02.880 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/19' 2023-10-30T12:50:02.880 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/15' 2023-10-30T12:50:02.880 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/31' 2023-10-30T12:50:02.881 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/36' 2023-10-30T12:50:02.881 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/20' 2023-10-30T12:50:02.881 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/12' 2023-10-30T12:50:02.881 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/39' 2023-10-30T12:50:02.881 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/10' 2023-10-30T12:50:02.882 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/34' 2023-10-30T12:50:02.882 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/38' 2023-10-30T12:50:02.882 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/11' 2023-10-30T12:50:02.882 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/16' 2023-10-30T12:50:02.882 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/32' 2023-10-30T12:50:02.883 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/30' 2023-10-30T12:50:02.883 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/14' 2023-10-30T12:50:02.883 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/18' 2023-10-30T12:50:02.883 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/13' 2023-10-30T12:50:02.883 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/33' 2023-10-30T12:50:02.884 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/17' 2023-10-30T12:50:02.884 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/35' 2023-10-30T12:50:02.884 INFO:tasks.workunit.client.0.smithi008.stdout:removed './106/37' 2023-10-30T12:50:02.884 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './106' 2023-10-30T12:50:02.885 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/19' 2023-10-30T12:50:02.885 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/15' 2023-10-30T12:50:02.885 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/31' 2023-10-30T12:50:02.885 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/36' 2023-10-30T12:50:02.885 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/20' 2023-10-30T12:50:02.886 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/12' 2023-10-30T12:50:02.886 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/39' 2023-10-30T12:50:02.886 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/10' 2023-10-30T12:50:02.886 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/34' 2023-10-30T12:50:02.886 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/38' 2023-10-30T12:50:02.887 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/11' 2023-10-30T12:50:02.887 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/16' 2023-10-30T12:50:02.887 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/32' 2023-10-30T12:50:02.887 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/30' 2023-10-30T12:50:02.887 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/14' 2023-10-30T12:50:02.888 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/18' 2023-10-30T12:50:02.888 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/13' 2023-10-30T12:50:02.888 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/33' 2023-10-30T12:50:02.888 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/17' 2023-10-30T12:50:02.888 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/35' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed './107/37' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './107' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/19' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/15' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/31' 2023-10-30T12:50:02.889 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/36' 2023-10-30T12:50:02.890 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/20' 2023-10-30T12:50:02.890 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/12' 2023-10-30T12:50:02.890 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/39' 2023-10-30T12:50:02.890 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/10' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/34' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/38' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/11' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/16' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/32' 2023-10-30T12:50:02.891 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/30' 2023-10-30T12:50:02.892 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/14' 2023-10-30T12:50:02.892 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/18' 2023-10-30T12:50:02.892 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/13' 2023-10-30T12:50:02.892 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/33' 2023-10-30T12:50:02.892 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/17' 2023-10-30T12:50:02.893 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/35' 2023-10-30T12:50:02.893 INFO:tasks.workunit.client.0.smithi008.stdout:removed './108/37' 2023-10-30T12:50:02.893 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './108' 2023-10-30T12:50:02.893 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/19' 2023-10-30T12:50:02.893 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/15' 2023-10-30T12:50:02.894 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/31' 2023-10-30T12:50:02.894 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/36' 2023-10-30T12:50:02.894 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/20' 2023-10-30T12:50:02.894 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/12' 2023-10-30T12:50:02.894 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/39' 2023-10-30T12:50:02.895 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/10' 2023-10-30T12:50:02.895 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/34' 2023-10-30T12:50:02.895 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/38' 2023-10-30T12:50:02.895 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/11' 2023-10-30T12:50:02.895 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/16' 2023-10-30T12:50:02.896 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/32' 2023-10-30T12:50:02.896 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/30' 2023-10-30T12:50:06.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/14' 2023-10-30T12:50:06.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/18' 2023-10-30T12:50:06.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/13' 2023-10-30T12:50:06.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/33' 2023-10-30T12:50:06.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/17' 2023-10-30T12:50:06.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/35' 2023-10-30T12:50:06.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './109/37' 2023-10-30T12:50:06.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './109' 2023-10-30T12:50:06.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/19' 2023-10-30T12:50:06.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/15' 2023-10-30T12:50:06.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/31' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/36' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/20' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/12' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/39' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/10' 2023-10-30T12:50:06.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/34' 2023-10-30T12:50:06.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/38' 2023-10-30T12:50:06.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/11' 2023-10-30T12:50:06.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/16' 2023-10-30T12:50:06.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/32' 2023-10-30T12:50:06.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/30' 2023-10-30T12:50:06.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/14' 2023-10-30T12:50:06.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/18' 2023-10-30T12:50:06.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/13' 2023-10-30T12:50:06.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/33' 2023-10-30T12:50:06.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/17' 2023-10-30T12:50:06.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/35' 2023-10-30T12:50:06.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './110/37' 2023-10-30T12:50:06.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './110' 2023-10-30T12:50:06.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/19' 2023-10-30T12:50:06.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/15' 2023-10-30T12:50:06.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/31' 2023-10-30T12:50:06.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/36' 2023-10-30T12:50:06.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/20' 2023-10-30T12:50:06.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/12' 2023-10-30T12:50:06.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/39' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/10' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/34' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/38' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/11' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/16' 2023-10-30T12:50:06.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/32' 2023-10-30T12:50:06.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/30' 2023-10-30T12:50:06.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/14' 2023-10-30T12:50:06.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/18' 2023-10-30T12:50:06.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/13' 2023-10-30T12:50:06.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/33' 2023-10-30T12:50:06.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/17' 2023-10-30T12:50:06.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/35' 2023-10-30T12:50:06.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './111/37' 2023-10-30T12:50:06.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './111' 2023-10-30T12:50:06.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/19' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/15' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/31' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/36' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/20' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/12' 2023-10-30T12:50:06.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/39' 2023-10-30T12:50:06.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/10' 2023-10-30T12:50:06.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/34' 2023-10-30T12:50:06.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/38' 2023-10-30T12:50:06.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/11' 2023-10-30T12:50:06.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/16' 2023-10-30T12:50:06.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/32' 2023-10-30T12:50:06.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/30' 2023-10-30T12:50:06.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/14' 2023-10-30T12:50:06.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/18' 2023-10-30T12:50:06.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/13' 2023-10-30T12:50:06.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/33' 2023-10-30T12:50:06.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/17' 2023-10-30T12:50:06.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/35' 2023-10-30T12:50:06.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './112/37' 2023-10-30T12:50:06.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './112' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/19' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/15' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/31' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/36' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/20' 2023-10-30T12:50:06.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/12' 2023-10-30T12:50:06.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/39' 2023-10-30T12:50:06.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/10' 2023-10-30T12:50:06.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/34' 2023-10-30T12:50:06.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/38' 2023-10-30T12:50:06.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/11' 2023-10-30T12:50:06.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/16' 2023-10-30T12:50:06.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/32' 2023-10-30T12:50:06.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/30' 2023-10-30T12:50:06.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/14' 2023-10-30T12:50:06.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/18' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/13' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/33' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/17' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/35' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './113/37' 2023-10-30T12:50:06.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './113' 2023-10-30T12:50:06.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/19' 2023-10-30T12:50:06.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/15' 2023-10-30T12:50:06.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/31' 2023-10-30T12:50:06.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/36' 2023-10-30T12:50:06.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/20' 2023-10-30T12:50:06.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/12' 2023-10-30T12:50:06.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/39' 2023-10-30T12:50:06.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/10' 2023-10-30T12:50:06.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/34' 2023-10-30T12:50:06.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/38' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/11' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/16' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/32' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/30' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/14' 2023-10-30T12:50:06.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/18' 2023-10-30T12:50:06.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/13' 2023-10-30T12:50:06.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/33' 2023-10-30T12:50:06.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/17' 2023-10-30T12:50:06.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/35' 2023-10-30T12:50:06.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './114/37' 2023-10-30T12:50:06.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './114' 2023-10-30T12:50:06.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/19' 2023-10-30T12:50:06.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/15' 2023-10-30T12:50:06.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/31' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/36' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/20' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/12' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/39' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/10' 2023-10-30T12:50:06.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/34' 2023-10-30T12:50:06.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/38' 2023-10-30T12:50:06.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/11' 2023-10-30T12:50:06.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/16' 2023-10-30T12:50:06.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/32' 2023-10-30T12:50:06.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/30' 2023-10-30T12:50:06.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/14' 2023-10-30T12:50:06.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/18' 2023-10-30T12:50:06.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/13' 2023-10-30T12:50:06.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/33' 2023-10-30T12:50:06.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/17' 2023-10-30T12:50:06.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/35' 2023-10-30T12:50:06.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './115/37' 2023-10-30T12:50:06.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './115' 2023-10-30T12:50:06.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/19' 2023-10-30T12:50:06.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/15' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/31' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/36' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/20' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/12' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/39' 2023-10-30T12:50:06.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/10' 2023-10-30T12:50:06.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/34' 2023-10-30T12:50:06.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/38' 2023-10-30T12:50:06.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/11' 2023-10-30T12:50:06.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/16' 2023-10-30T12:50:06.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/32' 2023-10-30T12:50:06.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/30' 2023-10-30T12:50:06.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/14' 2023-10-30T12:50:06.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/18' 2023-10-30T12:50:06.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/13' 2023-10-30T12:50:06.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/33' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/17' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/35' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './116/37' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './116' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/19' 2023-10-30T12:50:06.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/15' 2023-10-30T12:50:06.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/31' 2023-10-30T12:50:06.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/36' 2023-10-30T12:50:06.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/20' 2023-10-30T12:50:06.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/12' 2023-10-30T12:50:06.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/39' 2023-10-30T12:50:06.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/10' 2023-10-30T12:50:06.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/34' 2023-10-30T12:50:06.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/38' 2023-10-30T12:50:06.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/11' 2023-10-30T12:50:06.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/16' 2023-10-30T12:50:06.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/32' 2023-10-30T12:50:06.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/30' 2023-10-30T12:50:06.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/14' 2023-10-30T12:50:06.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/18' 2023-10-30T12:50:06.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/13' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/33' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/17' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/35' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './117/37' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './117' 2023-10-30T12:50:06.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/19' 2023-10-30T12:50:06.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/15' 2023-10-30T12:50:06.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/31' 2023-10-30T12:50:06.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/36' 2023-10-30T12:50:06.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/20' 2023-10-30T12:50:06.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/12' 2023-10-30T12:50:06.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/39' 2023-10-30T12:50:06.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/10' 2023-10-30T12:50:06.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/34' 2023-10-30T12:50:06.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/38' 2023-10-30T12:50:06.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/11' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/16' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/32' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/30' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/14' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/18' 2023-10-30T12:50:06.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/13' 2023-10-30T12:50:06.415 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/33' 2023-10-30T12:50:06.415 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/17' 2023-10-30T12:50:06.415 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/35' 2023-10-30T12:50:06.415 INFO:tasks.workunit.client.0.smithi008.stdout:removed './118/37' 2023-10-30T12:50:06.415 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './118' 2023-10-30T12:50:06.416 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/19' 2023-10-30T12:50:06.416 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/15' 2023-10-30T12:50:06.416 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/31' 2023-10-30T12:50:06.416 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/36' 2023-10-30T12:50:06.416 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/20' 2023-10-30T12:50:06.417 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/12' 2023-10-30T12:50:14.933 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:14.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:15.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/39' 2023-10-30T12:50:15.338 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/10' 2023-10-30T12:50:15.338 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/34' 2023-10-30T12:50:15.338 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/38' 2023-10-30T12:50:15.338 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/11' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/16' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/32' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/30' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/14' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/18' 2023-10-30T12:50:15.339 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/13' 2023-10-30T12:50:15.340 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/33' 2023-10-30T12:50:15.340 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/17' 2023-10-30T12:50:15.340 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/35' 2023-10-30T12:50:15.340 INFO:tasks.workunit.client.0.smithi008.stdout:removed './119/37' 2023-10-30T12:50:15.340 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './119' 2023-10-30T12:50:15.341 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/19' 2023-10-30T12:50:15.341 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/15' 2023-10-30T12:50:15.341 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/31' 2023-10-30T12:50:15.341 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/36' 2023-10-30T12:50:15.341 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/20' 2023-10-30T12:50:15.342 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/12' 2023-10-30T12:50:15.342 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/39' 2023-10-30T12:50:15.342 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/10' 2023-10-30T12:50:15.342 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/34' 2023-10-30T12:50:15.342 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/38' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/11' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/16' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/32' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/30' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/14' 2023-10-30T12:50:15.343 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/18' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/13' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/33' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/17' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/35' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed './120/37' 2023-10-30T12:50:15.344 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './120' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/19' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/15' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/31' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/36' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/20' 2023-10-30T12:50:15.345 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/12' 2023-10-30T12:50:15.346 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/39' 2023-10-30T12:50:15.346 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/10' 2023-10-30T12:50:15.346 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/34' 2023-10-30T12:50:15.346 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/38' 2023-10-30T12:50:15.347 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/11' 2023-10-30T12:50:15.347 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/16' 2023-10-30T12:50:15.347 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/32' 2023-10-30T12:50:15.347 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/30' 2023-10-30T12:50:15.347 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/14' 2023-10-30T12:50:15.348 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/18' 2023-10-30T12:50:15.348 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/13' 2023-10-30T12:50:15.348 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/33' 2023-10-30T12:50:15.348 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/17' 2023-10-30T12:50:15.348 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/35' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed './121/37' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './121' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/19' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/15' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/31' 2023-10-30T12:50:15.349 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/36' 2023-10-30T12:50:15.350 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/20' 2023-10-30T12:50:15.350 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/12' 2023-10-30T12:50:15.350 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/39' 2023-10-30T12:50:15.350 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/10' 2023-10-30T12:50:15.350 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/34' 2023-10-30T12:50:15.351 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/38' 2023-10-30T12:50:15.351 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/11' 2023-10-30T12:50:15.351 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/16' 2023-10-30T12:50:15.351 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/32' 2023-10-30T12:50:15.351 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/30' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/14' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/18' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/13' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/33' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/17' 2023-10-30T12:50:15.352 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/35' 2023-10-30T12:50:15.353 INFO:tasks.workunit.client.0.smithi008.stdout:removed './122/37' 2023-10-30T12:50:15.353 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './122' 2023-10-30T12:50:15.353 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/19' 2023-10-30T12:50:15.353 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/15' 2023-10-30T12:50:15.353 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/31' 2023-10-30T12:50:15.354 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/36' 2023-10-30T12:50:15.354 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/20' 2023-10-30T12:50:15.354 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/12' 2023-10-30T12:50:15.354 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/39' 2023-10-30T12:50:15.354 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/10' 2023-10-30T12:50:15.355 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/34' 2023-10-30T12:50:15.355 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/38' 2023-10-30T12:50:15.355 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/11' 2023-10-30T12:50:15.355 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/16' 2023-10-30T12:50:15.355 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/32' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/30' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/14' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/18' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/13' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/33' 2023-10-30T12:50:15.356 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/17' 2023-10-30T12:50:15.357 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/35' 2023-10-30T12:50:15.357 INFO:tasks.workunit.client.0.smithi008.stdout:removed './123/37' 2023-10-30T12:50:15.357 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './123' 2023-10-30T12:50:15.357 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/19' 2023-10-30T12:50:15.358 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/15' 2023-10-30T12:50:15.358 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/31' 2023-10-30T12:50:15.358 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/36' 2023-10-30T12:50:15.358 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/20' 2023-10-30T12:50:15.358 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/12' 2023-10-30T12:50:15.359 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/39' 2023-10-30T12:50:15.359 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/10' 2023-10-30T12:50:15.359 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/34' 2023-10-30T12:50:15.359 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/38' 2023-10-30T12:50:15.359 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/11' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/16' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/32' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/30' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/14' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/18' 2023-10-30T12:50:15.360 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/13' 2023-10-30T12:50:15.361 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/33' 2023-10-30T12:50:15.361 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/17' 2023-10-30T12:50:15.361 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/35' 2023-10-30T12:50:15.361 INFO:tasks.workunit.client.0.smithi008.stdout:removed './124/37' 2023-10-30T12:50:15.361 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './124' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/19' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/15' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/31' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/36' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/20' 2023-10-30T12:50:15.362 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/12' 2023-10-30T12:50:15.363 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/39' 2023-10-30T12:50:15.363 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/10' 2023-10-30T12:50:15.363 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/34' 2023-10-30T12:50:15.363 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/38' 2023-10-30T12:50:15.363 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/11' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/16' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/32' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/30' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/14' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/18' 2023-10-30T12:50:15.364 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/13' 2023-10-30T12:50:15.365 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/33' 2023-10-30T12:50:15.365 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/17' 2023-10-30T12:50:15.365 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/35' 2023-10-30T12:50:15.365 INFO:tasks.workunit.client.0.smithi008.stdout:removed './125/37' 2023-10-30T12:50:15.365 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './125' 2023-10-30T12:50:15.366 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/19' 2023-10-30T12:50:15.366 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/15' 2023-10-30T12:50:15.366 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/31' 2023-10-30T12:50:15.366 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/36' 2023-10-30T12:50:15.366 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/20' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/12' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/39' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/10' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/34' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/38' 2023-10-30T12:50:15.367 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/11' 2023-10-30T12:50:15.368 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/16' 2023-10-30T12:50:15.368 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/32' 2023-10-30T12:50:15.368 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/30' 2023-10-30T12:50:15.368 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/14' 2023-10-30T12:50:15.368 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/18' 2023-10-30T12:50:15.369 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/13' 2023-10-30T12:50:15.369 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/33' 2023-10-30T12:50:15.369 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/17' 2023-10-30T12:50:15.369 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/35' 2023-10-30T12:50:15.369 INFO:tasks.workunit.client.0.smithi008.stdout:removed './126/37' 2023-10-30T12:50:15.370 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './126' 2023-10-30T12:50:15.370 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/19' 2023-10-30T12:50:15.370 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/15' 2023-10-30T12:50:15.370 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/31' 2023-10-30T12:50:15.370 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/36' 2023-10-30T12:50:15.371 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/20' 2023-10-30T12:50:15.371 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/12' 2023-10-30T12:50:15.371 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/39' 2023-10-30T12:50:15.371 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/10' 2023-10-30T12:50:15.371 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/34' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/38' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/11' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/16' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/32' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/30' 2023-10-30T12:50:15.372 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/14' 2023-10-30T12:50:15.373 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/18' 2023-10-30T12:50:15.373 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/13' 2023-10-30T12:50:15.373 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/33' 2023-10-30T12:50:15.373 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/17' 2023-10-30T12:50:15.373 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/35' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed './127/37' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './127' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/19' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/15' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/31' 2023-10-30T12:50:15.374 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/36' 2023-10-30T12:50:15.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/20' 2023-10-30T12:50:15.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/12' 2023-10-30T12:50:15.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/39' 2023-10-30T12:50:15.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/10' 2023-10-30T12:50:15.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/34' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/38' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/11' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/16' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/32' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/30' 2023-10-30T12:50:15.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/14' 2023-10-30T12:50:15.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/18' 2023-10-30T12:50:15.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/13' 2023-10-30T12:50:15.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/33' 2023-10-30T12:50:15.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/17' 2023-10-30T12:50:15.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/35' 2023-10-30T12:50:17.754 INFO:tasks.workunit.client.0.smithi008.stdout:removed './128/37' 2023-10-30T12:50:17.755 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './128' 2023-10-30T12:50:17.755 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/19' 2023-10-30T12:50:17.755 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/15' 2023-10-30T12:50:17.755 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/31' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/36' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/20' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/12' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/39' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/10' 2023-10-30T12:50:17.756 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/34' 2023-10-30T12:50:17.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/38' 2023-10-30T12:50:17.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/11' 2023-10-30T12:50:17.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/16' 2023-10-30T12:50:17.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/32' 2023-10-30T12:50:17.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/30' 2023-10-30T12:50:17.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/14' 2023-10-30T12:50:17.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/18' 2023-10-30T12:50:17.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/13' 2023-10-30T12:50:17.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/33' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/17' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/35' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './129/37' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './129' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/19' 2023-10-30T12:50:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/15' 2023-10-30T12:50:17.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/31' 2023-10-30T12:50:17.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/36' 2023-10-30T12:50:17.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/20' 2023-10-30T12:50:17.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/12' 2023-10-30T12:50:17.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/39' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/10' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/34' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/38' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/11' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/16' 2023-10-30T12:50:17.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/32' 2023-10-30T12:50:17.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/30' 2023-10-30T12:50:17.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/14' 2023-10-30T12:50:17.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/18' 2023-10-30T12:50:17.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/13' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/33' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/17' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/35' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './130/37' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './130' 2023-10-30T12:50:17.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/19' 2023-10-30T12:50:17.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/15' 2023-10-30T12:50:17.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/31' 2023-10-30T12:50:17.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/36' 2023-10-30T12:50:17.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/20' 2023-10-30T12:50:17.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/12' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/39' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/10' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/34' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/38' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/11' 2023-10-30T12:50:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/16' 2023-10-30T12:50:17.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/32' 2023-10-30T12:50:17.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/30' 2023-10-30T12:50:17.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/14' 2023-10-30T12:50:17.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/18' 2023-10-30T12:50:17.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/13' 2023-10-30T12:50:17.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/33' 2023-10-30T12:50:17.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/17' 2023-10-30T12:50:17.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/35' 2023-10-30T12:50:17.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './131/37' 2023-10-30T12:50:17.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './131' 2023-10-30T12:50:17.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/19' 2023-10-30T12:50:17.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/15' 2023-10-30T12:50:17.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/31' 2023-10-30T12:50:17.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/36' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/20' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/12' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/39' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/10' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/34' 2023-10-30T12:50:17.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/38' 2023-10-30T12:50:17.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/11' 2023-10-30T12:50:17.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/16' 2023-10-30T12:50:17.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/32' 2023-10-30T12:50:17.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/30' 2023-10-30T12:50:17.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/14' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/18' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/13' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/33' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/17' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/35' 2023-10-30T12:50:17.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './132/37' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './132' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/19' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/15' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/31' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/36' 2023-10-30T12:50:17.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/20' 2023-10-30T12:50:17.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/12' 2023-10-30T12:50:17.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/39' 2023-10-30T12:50:17.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/10' 2023-10-30T12:50:17.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/34' 2023-10-30T12:50:17.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/38' 2023-10-30T12:50:17.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/11' 2023-10-30T12:50:17.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/16' 2023-10-30T12:50:17.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/32' 2023-10-30T12:50:17.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/30' 2023-10-30T12:50:17.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/14' 2023-10-30T12:50:17.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/18' 2023-10-30T12:50:17.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/13' 2023-10-30T12:50:17.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/33' 2023-10-30T12:50:17.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/17' 2023-10-30T12:50:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/35' 2023-10-30T12:50:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './133/37' 2023-10-30T12:50:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './133' 2023-10-30T12:50:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/19' 2023-10-30T12:50:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/15' 2023-10-30T12:50:17.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/31' 2023-10-30T12:50:17.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/36' 2023-10-30T12:50:17.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/20' 2023-10-30T12:50:17.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/12' 2023-10-30T12:50:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/39' 2023-10-30T12:50:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/10' 2023-10-30T12:50:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/34' 2023-10-30T12:50:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/38' 2023-10-30T12:50:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/11' 2023-10-30T12:50:17.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/16' 2023-10-30T12:50:17.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/32' 2023-10-30T12:50:17.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/30' 2023-10-30T12:50:17.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/14' 2023-10-30T12:50:17.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/18' 2023-10-30T12:50:17.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/13' 2023-10-30T12:50:17.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/33' 2023-10-30T12:50:17.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/17' 2023-10-30T12:50:17.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/35' 2023-10-30T12:50:17.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './134/37' 2023-10-30T12:50:17.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './134' 2023-10-30T12:50:17.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/19' 2023-10-30T12:50:17.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/15' 2023-10-30T12:50:17.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/31' 2023-10-30T12:50:17.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/36' 2023-10-30T12:50:17.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/20' 2023-10-30T12:50:17.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/12' 2023-10-30T12:50:17.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/39' 2023-10-30T12:50:17.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/10' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/34' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/38' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/11' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/16' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/32' 2023-10-30T12:50:17.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/30' 2023-10-30T12:50:17.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/14' 2023-10-30T12:50:17.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/18' 2023-10-30T12:50:17.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/13' 2023-10-30T12:50:17.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/33' 2023-10-30T12:50:17.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/17' 2023-10-30T12:50:17.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/35' 2023-10-30T12:50:17.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './135/37' 2023-10-30T12:50:17.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './135' 2023-10-30T12:50:17.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/19' 2023-10-30T12:50:17.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/15' 2023-10-30T12:50:17.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/31' 2023-10-30T12:50:17.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/36' 2023-10-30T12:50:17.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/20' 2023-10-30T12:50:17.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/12' 2023-10-30T12:50:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/39' 2023-10-30T12:50:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/10' 2023-10-30T12:50:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/34' 2023-10-30T12:50:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/38' 2023-10-30T12:50:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/11' 2023-10-30T12:50:17.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/16' 2023-10-30T12:50:17.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/32' 2023-10-30T12:50:17.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/30' 2023-10-30T12:50:17.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/14' 2023-10-30T12:50:17.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/18' 2023-10-30T12:50:17.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/13' 2023-10-30T12:50:17.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/33' 2023-10-30T12:50:17.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/17' 2023-10-30T12:50:17.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/35' 2023-10-30T12:50:17.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './136/37' 2023-10-30T12:50:17.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './136' 2023-10-30T12:50:17.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/19' 2023-10-30T12:50:17.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/15' 2023-10-30T12:50:17.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/31' 2023-10-30T12:50:17.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/36' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/20' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/12' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/39' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/10' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/34' 2023-10-30T12:50:17.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/38' 2023-10-30T12:50:17.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/11' 2023-10-30T12:50:17.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/16' 2023-10-30T12:50:17.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/32' 2023-10-30T12:50:17.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/30' 2023-10-30T12:50:17.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/14' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/18' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/13' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/33' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/17' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/35' 2023-10-30T12:50:17.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './137/37' 2023-10-30T12:50:17.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './137' 2023-10-30T12:50:17.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/19' 2023-10-30T12:50:17.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/15' 2023-10-30T12:50:17.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/31' 2023-10-30T12:50:17.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/36' 2023-10-30T12:50:17.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/20' 2023-10-30T12:50:17.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/12' 2023-10-30T12:50:17.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/39' 2023-10-30T12:50:17.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/10' 2023-10-30T12:50:17.796 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/34' 2023-10-30T12:50:17.796 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/38' 2023-10-30T12:50:17.796 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/11' 2023-10-30T12:50:17.796 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/16' 2023-10-30T12:50:19.092 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/32' 2023-10-30T12:50:19.092 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/30' 2023-10-30T12:50:19.093 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/14' 2023-10-30T12:50:19.093 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/18' 2023-10-30T12:50:19.093 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/13' 2023-10-30T12:50:19.093 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/33' 2023-10-30T12:50:19.093 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/17' 2023-10-30T12:50:19.094 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/35' 2023-10-30T12:50:19.094 INFO:tasks.workunit.client.0.smithi008.stdout:removed './138/37' 2023-10-30T12:50:19.094 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './138' 2023-10-30T12:50:19.094 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/19' 2023-10-30T12:50:19.094 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/15' 2023-10-30T12:50:19.095 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/31' 2023-10-30T12:50:19.095 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/36' 2023-10-30T12:50:19.095 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/20' 2023-10-30T12:50:19.095 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/12' 2023-10-30T12:50:19.095 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/39' 2023-10-30T12:50:19.096 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/10' 2023-10-30T12:50:19.096 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/34' 2023-10-30T12:50:19.096 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/38' 2023-10-30T12:50:19.096 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/11' 2023-10-30T12:50:19.096 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/16' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/32' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/30' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/14' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/18' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/13' 2023-10-30T12:50:19.097 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/33' 2023-10-30T12:50:19.098 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/17' 2023-10-30T12:50:19.098 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/35' 2023-10-30T12:50:19.098 INFO:tasks.workunit.client.0.smithi008.stdout:removed './139/37' 2023-10-30T12:50:19.098 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './139' 2023-10-30T12:50:19.098 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/19' 2023-10-30T12:50:19.099 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/15' 2023-10-30T12:50:19.099 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/31' 2023-10-30T12:50:19.099 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/36' 2023-10-30T12:50:19.099 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/20' 2023-10-30T12:50:19.099 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/12' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/39' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/10' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/34' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/38' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/11' 2023-10-30T12:50:19.100 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/16' 2023-10-30T12:50:19.101 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/32' 2023-10-30T12:50:19.101 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/30' 2023-10-30T12:50:19.101 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/14' 2023-10-30T12:50:19.101 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/18' 2023-10-30T12:50:19.101 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/13' 2023-10-30T12:50:19.102 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/33' 2023-10-30T12:50:19.102 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/17' 2023-10-30T12:50:19.102 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/35' 2023-10-30T12:50:19.102 INFO:tasks.workunit.client.0.smithi008.stdout:removed './140/37' 2023-10-30T12:50:19.102 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './140' 2023-10-30T12:50:19.103 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/19' 2023-10-30T12:50:19.103 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/15' 2023-10-30T12:50:19.103 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/31' 2023-10-30T12:50:19.103 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/36' 2023-10-30T12:50:19.103 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/20' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/12' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/39' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/10' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/34' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/38' 2023-10-30T12:50:19.104 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/11' 2023-10-30T12:50:19.105 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/16' 2023-10-30T12:50:19.105 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/32' 2023-10-30T12:50:19.105 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/30' 2023-10-30T12:50:19.105 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/14' 2023-10-30T12:50:19.105 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/18' 2023-10-30T12:50:19.106 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/13' 2023-10-30T12:50:19.106 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/33' 2023-10-30T12:50:19.106 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/17' 2023-10-30T12:50:19.106 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/35' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed './141/37' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './141' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/19' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/15' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/31' 2023-10-30T12:50:19.107 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/36' 2023-10-30T12:50:19.108 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/20' 2023-10-30T12:50:19.108 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/12' 2023-10-30T12:50:19.108 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/39' 2023-10-30T12:50:19.108 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/10' 2023-10-30T12:50:19.108 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/34' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/38' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/11' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/16' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/32' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/30' 2023-10-30T12:50:19.109 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/14' 2023-10-30T12:50:19.110 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/18' 2023-10-30T12:50:19.110 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/13' 2023-10-30T12:50:19.110 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/33' 2023-10-30T12:50:19.110 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/17' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/35' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed './142/37' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './142' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/19' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/15' 2023-10-30T12:50:19.111 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/31' 2023-10-30T12:50:19.112 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/36' 2023-10-30T12:50:19.112 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/20' 2023-10-30T12:50:19.112 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/12' 2023-10-30T12:50:19.112 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/39' 2023-10-30T12:50:19.112 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/10' 2023-10-30T12:50:19.113 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/34' 2023-10-30T12:50:19.113 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/38' 2023-10-30T12:50:19.113 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/11' 2023-10-30T12:50:19.113 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/16' 2023-10-30T12:50:19.113 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/32' 2023-10-30T12:50:19.114 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/30' 2023-10-30T12:50:19.114 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/14' 2023-10-30T12:50:19.114 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/18' 2023-10-30T12:50:19.114 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/13' 2023-10-30T12:50:19.114 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/33' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/17' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/35' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed './143/37' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './143' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/19' 2023-10-30T12:50:19.115 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/15' 2023-10-30T12:50:19.116 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/31' 2023-10-30T12:50:19.116 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/36' 2023-10-30T12:50:19.116 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/20' 2023-10-30T12:50:19.116 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/12' 2023-10-30T12:50:19.116 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/39' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/10' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/34' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/38' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/11' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/16' 2023-10-30T12:50:19.117 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/32' 2023-10-30T12:50:19.118 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/30' 2023-10-30T12:50:19.118 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/14' 2023-10-30T12:50:19.118 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/18' 2023-10-30T12:50:19.118 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/13' 2023-10-30T12:50:19.118 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/33' 2023-10-30T12:50:19.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/17' 2023-10-30T12:50:19.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/35' 2023-10-30T12:50:19.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed './144/37' 2023-10-30T12:50:19.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './144' 2023-10-30T12:50:19.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/19' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/15' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/31' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/36' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/20' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/12' 2023-10-30T12:50:19.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/39' 2023-10-30T12:50:19.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/10' 2023-10-30T12:50:19.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/34' 2023-10-30T12:50:19.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/38' 2023-10-30T12:50:19.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/11' 2023-10-30T12:50:19.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/16' 2023-10-30T12:50:19.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/32' 2023-10-30T12:50:19.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/30' 2023-10-30T12:50:19.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/14' 2023-10-30T12:50:19.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/18' 2023-10-30T12:50:19.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/13' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/33' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/17' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/35' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './145/37' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './145' 2023-10-30T12:50:19.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/19' 2023-10-30T12:50:19.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/15' 2023-10-30T12:50:19.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/31' 2023-10-30T12:50:19.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/36' 2023-10-30T12:50:19.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/20' 2023-10-30T12:50:19.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/12' 2023-10-30T12:50:19.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/39' 2023-10-30T12:50:19.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/10' 2023-10-30T12:50:19.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/34' 2023-10-30T12:50:19.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/38' 2023-10-30T12:50:19.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/11' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/16' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/32' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/30' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/14' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/18' 2023-10-30T12:50:19.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/13' 2023-10-30T12:50:19.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/33' 2023-10-30T12:50:19.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/17' 2023-10-30T12:50:19.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/35' 2023-10-30T12:50:19.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './146/37' 2023-10-30T12:50:19.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './146' 2023-10-30T12:50:19.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/19' 2023-10-30T12:50:19.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/15' 2023-10-30T12:50:19.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/31' 2023-10-30T12:50:19.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/36' 2023-10-30T12:50:19.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/20' 2023-10-30T12:50:19.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/12' 2023-10-30T12:50:19.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/39' 2023-10-30T12:50:19.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/10' 2023-10-30T12:50:19.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/34' 2023-10-30T12:50:19.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/38' 2023-10-30T12:50:19.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/11' 2023-10-30T12:50:19.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/16' 2023-10-30T12:50:19.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/32' 2023-10-30T12:50:19.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/30' 2023-10-30T12:50:19.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/14' 2023-10-30T12:50:19.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/18' 2023-10-30T12:50:19.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/13' 2023-10-30T12:50:19.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/33' 2023-10-30T12:50:19.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/17' 2023-10-30T12:50:19.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/35' 2023-10-30T12:50:19.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './147/37' 2023-10-30T12:50:19.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './147' 2023-10-30T12:50:19.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/19' 2023-10-30T12:50:19.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/15' 2023-10-30T12:50:19.134 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/31' 2023-10-30T12:50:19.134 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/36' 2023-10-30T12:50:21.429 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/20' 2023-10-30T12:50:21.429 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/12' 2023-10-30T12:50:21.429 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/39' 2023-10-30T12:50:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/10' 2023-10-30T12:50:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/34' 2023-10-30T12:50:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/38' 2023-10-30T12:50:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/11' 2023-10-30T12:50:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/16' 2023-10-30T12:50:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/32' 2023-10-30T12:50:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/30' 2023-10-30T12:50:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/14' 2023-10-30T12:50:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/18' 2023-10-30T12:50:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/13' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/33' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/17' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/35' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed './148/37' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './148' 2023-10-30T12:50:21.432 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/19' 2023-10-30T12:50:21.433 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/15' 2023-10-30T12:50:21.433 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/31' 2023-10-30T12:50:21.433 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/36' 2023-10-30T12:50:21.433 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/20' 2023-10-30T12:50:21.433 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/12' 2023-10-30T12:50:21.434 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/39' 2023-10-30T12:50:21.434 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/10' 2023-10-30T12:50:21.434 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/34' 2023-10-30T12:50:21.434 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/38' 2023-10-30T12:50:21.434 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/11' 2023-10-30T12:50:21.435 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/16' 2023-10-30T12:50:21.435 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/32' 2023-10-30T12:50:21.435 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/30' 2023-10-30T12:50:21.435 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/14' 2023-10-30T12:50:21.435 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/18' 2023-10-30T12:50:21.436 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/13' 2023-10-30T12:50:21.436 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/33' 2023-10-30T12:50:21.436 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/17' 2023-10-30T12:50:21.436 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/35' 2023-10-30T12:50:21.436 INFO:tasks.workunit.client.0.smithi008.stdout:removed './149/37' 2023-10-30T12:50:21.437 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './149' 2023-10-30T12:50:21.437 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/19' 2023-10-30T12:50:21.437 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/15' 2023-10-30T12:50:21.437 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/31' 2023-10-30T12:50:21.437 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/36' 2023-10-30T12:50:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/20' 2023-10-30T12:50:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/12' 2023-10-30T12:50:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/39' 2023-10-30T12:50:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/10' 2023-10-30T12:50:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/34' 2023-10-30T12:50:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/38' 2023-10-30T12:50:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/11' 2023-10-30T12:50:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/16' 2023-10-30T12:50:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/32' 2023-10-30T12:50:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/30' 2023-10-30T12:50:21.440 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/14' 2023-10-30T12:50:21.440 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/18' 2023-10-30T12:50:21.440 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/13' 2023-10-30T12:50:21.440 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/33' 2023-10-30T12:50:21.441 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/17' 2023-10-30T12:50:21.441 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/35' 2023-10-30T12:50:21.441 INFO:tasks.workunit.client.0.smithi008.stdout:removed './150/37' 2023-10-30T12:50:21.441 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './150' 2023-10-30T12:50:21.441 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/19' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/15' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/31' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/36' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/20' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/12' 2023-10-30T12:50:21.442 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/39' 2023-10-30T12:50:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/10' 2023-10-30T12:50:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/34' 2023-10-30T12:50:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/38' 2023-10-30T12:50:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/11' 2023-10-30T12:50:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/16' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/32' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/30' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/14' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/18' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/13' 2023-10-30T12:50:21.444 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/33' 2023-10-30T12:50:21.445 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/17' 2023-10-30T12:50:21.445 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/35' 2023-10-30T12:50:21.445 INFO:tasks.workunit.client.0.smithi008.stdout:removed './151/37' 2023-10-30T12:50:21.445 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './151' 2023-10-30T12:50:21.445 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/19' 2023-10-30T12:50:21.446 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/15' 2023-10-30T12:50:21.446 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/31' 2023-10-30T12:50:21.446 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/36' 2023-10-30T12:50:21.446 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/20' 2023-10-30T12:50:21.447 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/12' 2023-10-30T12:50:21.447 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/39' 2023-10-30T12:50:21.447 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/10' 2023-10-30T12:50:21.447 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/34' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/38' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/11' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/16' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/32' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/30' 2023-10-30T12:50:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/14' 2023-10-30T12:50:21.449 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/18' 2023-10-30T12:50:21.449 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/13' 2023-10-30T12:50:21.449 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/33' 2023-10-30T12:50:21.449 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/17' 2023-10-30T12:50:21.449 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/35' 2023-10-30T12:50:21.450 INFO:tasks.workunit.client.0.smithi008.stdout:removed './152/37' 2023-10-30T12:50:21.450 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './152' 2023-10-30T12:50:21.450 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/19' 2023-10-30T12:50:21.450 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/15' 2023-10-30T12:50:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/31' 2023-10-30T12:50:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/36' 2023-10-30T12:50:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/20' 2023-10-30T12:50:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/12' 2023-10-30T12:50:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/39' 2023-10-30T12:50:21.452 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/10' 2023-10-30T12:50:21.452 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/34' 2023-10-30T12:50:21.452 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/38' 2023-10-30T12:50:21.452 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/11' 2023-10-30T12:50:21.452 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/16' 2023-10-30T12:50:21.453 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/32' 2023-10-30T12:50:21.453 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/30' 2023-10-30T12:50:21.453 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/14' 2023-10-30T12:50:21.453 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/18' 2023-10-30T12:50:21.453 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/13' 2023-10-30T12:50:21.454 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/33' 2023-10-30T12:50:21.454 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/17' 2023-10-30T12:50:21.454 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/35' 2023-10-30T12:50:21.454 INFO:tasks.workunit.client.0.smithi008.stdout:removed './153/37' 2023-10-30T12:50:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './153' 2023-10-30T12:50:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/19' 2023-10-30T12:50:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/15' 2023-10-30T12:50:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/31' 2023-10-30T12:50:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/36' 2023-10-30T12:50:21.456 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/20' 2023-10-30T12:50:21.456 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/12' 2023-10-30T12:50:21.456 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/39' 2023-10-30T12:50:21.456 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/10' 2023-10-30T12:50:21.456 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/34' 2023-10-30T12:50:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/38' 2023-10-30T12:50:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/11' 2023-10-30T12:50:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/16' 2023-10-30T12:50:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/32' 2023-10-30T12:50:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/30' 2023-10-30T12:50:21.458 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/14' 2023-10-30T12:50:21.458 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/18' 2023-10-30T12:50:21.458 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/13' 2023-10-30T12:50:21.458 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/33' 2023-10-30T12:50:21.458 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/17' 2023-10-30T12:50:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/35' 2023-10-30T12:50:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:removed './154/37' 2023-10-30T12:50:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './154' 2023-10-30T12:50:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/19' 2023-10-30T12:50:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/15' 2023-10-30T12:50:21.460 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/31' 2023-10-30T12:50:21.460 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/36' 2023-10-30T12:50:21.460 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/20' 2023-10-30T12:50:21.460 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/12' 2023-10-30T12:50:21.460 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/39' 2023-10-30T12:50:21.461 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/10' 2023-10-30T12:50:21.461 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/34' 2023-10-30T12:50:21.461 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/38' 2023-10-30T12:50:21.461 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/11' 2023-10-30T12:50:21.462 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/16' 2023-10-30T12:50:21.462 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/32' 2023-10-30T12:50:21.462 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/30' 2023-10-30T12:50:21.462 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/14' 2023-10-30T12:50:21.462 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/18' 2023-10-30T12:50:21.463 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/13' 2023-10-30T12:50:21.463 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/33' 2023-10-30T12:50:21.463 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/17' 2023-10-30T12:50:21.463 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/35' 2023-10-30T12:50:21.463 INFO:tasks.workunit.client.0.smithi008.stdout:removed './155/37' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './155' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/19' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/15' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/31' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/36' 2023-10-30T12:50:21.464 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/20' 2023-10-30T12:50:21.465 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/12' 2023-10-30T12:50:21.465 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/39' 2023-10-30T12:50:21.465 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/10' 2023-10-30T12:50:21.465 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/34' 2023-10-30T12:50:21.465 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/38' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/11' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/16' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/32' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/30' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/14' 2023-10-30T12:50:21.466 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/18' 2023-10-30T12:50:21.467 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/13' 2023-10-30T12:50:21.467 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/33' 2023-10-30T12:50:21.467 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/17' 2023-10-30T12:50:21.467 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/35' 2023-10-30T12:50:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:removed './156/37' 2023-10-30T12:50:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './156' 2023-10-30T12:50:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/19' 2023-10-30T12:50:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/15' 2023-10-30T12:50:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/31' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/36' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/20' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/12' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/39' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/10' 2023-10-30T12:50:21.469 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/34' 2023-10-30T12:50:21.470 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/38' 2023-10-30T12:50:21.470 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/11' 2023-10-30T12:50:21.470 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/16' 2023-10-30T12:50:21.470 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/32' 2023-10-30T12:50:21.470 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/30' 2023-10-30T12:50:21.471 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/14' 2023-10-30T12:50:21.471 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/18' 2023-10-30T12:50:21.471 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/13' 2023-10-30T12:50:21.471 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/33' 2023-10-30T12:50:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/17' 2023-10-30T12:50:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/35' 2023-10-30T12:50:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './157/37' 2023-10-30T12:50:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './157' 2023-10-30T12:50:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/19' 2023-10-30T12:50:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/15' 2023-10-30T12:50:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/31' 2023-10-30T12:50:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/36' 2023-10-30T12:50:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/20' 2023-10-30T12:50:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/12' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/39' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/10' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/34' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/38' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/11' 2023-10-30T12:50:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/16' 2023-10-30T12:50:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/32' 2023-10-30T12:50:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/30' 2023-10-30T12:50:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/14' 2023-10-30T12:50:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/18' 2023-10-30T12:50:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/13' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/33' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/17' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/35' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './158/37' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './158' 2023-10-30T12:50:23.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/19' 2023-10-30T12:50:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/15' 2023-10-30T12:50:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/31' 2023-10-30T12:50:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/36' 2023-10-30T12:50:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/20' 2023-10-30T12:50:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/12' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/39' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/10' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/34' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/38' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/11' 2023-10-30T12:50:23.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/16' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/32' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/30' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/14' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/18' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/13' 2023-10-30T12:50:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/33' 2023-10-30T12:50:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/17' 2023-10-30T12:50:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/35' 2023-10-30T12:50:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './159/37' 2023-10-30T12:50:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './159' 2023-10-30T12:50:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/19' 2023-10-30T12:50:23.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/15' 2023-10-30T12:50:23.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/31' 2023-10-30T12:50:23.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/36' 2023-10-30T12:50:23.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/20' 2023-10-30T12:50:23.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/12' 2023-10-30T12:50:23.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/39' 2023-10-30T12:50:23.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/10' 2023-10-30T12:50:23.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/34' 2023-10-30T12:50:23.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/38' 2023-10-30T12:50:23.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/11' 2023-10-30T12:50:23.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/16' 2023-10-30T12:50:23.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/32' 2023-10-30T12:50:23.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/30' 2023-10-30T12:50:23.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/14' 2023-10-30T12:50:23.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/18' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/13' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/33' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/17' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/35' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './160/37' 2023-10-30T12:50:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './160' 2023-10-30T12:50:23.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/19' 2023-10-30T12:50:23.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/15' 2023-10-30T12:50:23.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/31' 2023-10-30T12:50:23.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/36' 2023-10-30T12:50:23.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/20' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/12' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/39' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/10' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/34' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/38' 2023-10-30T12:50:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/11' 2023-10-30T12:50:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/16' 2023-10-30T12:50:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/32' 2023-10-30T12:50:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/30' 2023-10-30T12:50:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/14' 2023-10-30T12:50:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/18' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/13' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/33' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/17' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/35' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './161/37' 2023-10-30T12:50:23.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './161' 2023-10-30T12:50:23.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/19' 2023-10-30T12:50:23.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/15' 2023-10-30T12:50:23.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/31' 2023-10-30T12:50:23.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/36' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/20' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/12' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/39' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/10' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/34' 2023-10-30T12:50:23.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/38' 2023-10-30T12:50:23.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/11' 2023-10-30T12:50:23.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/16' 2023-10-30T12:50:23.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/32' 2023-10-30T12:50:23.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/30' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/14' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/18' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/13' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/33' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/17' 2023-10-30T12:50:23.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/35' 2023-10-30T12:50:23.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './162/37' 2023-10-30T12:50:23.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './162' 2023-10-30T12:50:23.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/19' 2023-10-30T12:50:23.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/15' 2023-10-30T12:50:23.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/31' 2023-10-30T12:50:23.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/36' 2023-10-30T12:50:23.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/20' 2023-10-30T12:50:23.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/12' 2023-10-30T12:50:23.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/39' 2023-10-30T12:50:23.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/10' 2023-10-30T12:50:23.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/34' 2023-10-30T12:50:23.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/38' 2023-10-30T12:50:23.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/11' 2023-10-30T12:50:23.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/16' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/32' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/30' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/14' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/18' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/13' 2023-10-30T12:50:23.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/33' 2023-10-30T12:50:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/17' 2023-10-30T12:50:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/35' 2023-10-30T12:50:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './163/37' 2023-10-30T12:50:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './163' 2023-10-30T12:50:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/19' 2023-10-30T12:50:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/15' 2023-10-30T12:50:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/31' 2023-10-30T12:50:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/36' 2023-10-30T12:50:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/20' 2023-10-30T12:50:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/12' 2023-10-30T12:50:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/39' 2023-10-30T12:50:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/10' 2023-10-30T12:50:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/34' 2023-10-30T12:50:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/38' 2023-10-30T12:50:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/11' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/16' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/32' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/30' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/14' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/18' 2023-10-30T12:50:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/13' 2023-10-30T12:50:23.182 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/33' 2023-10-30T12:50:23.182 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/17' 2023-10-30T12:50:23.182 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/35' 2023-10-30T12:50:23.182 INFO:tasks.workunit.client.0.smithi008.stdout:removed './164/37' 2023-10-30T12:50:23.182 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './164' 2023-10-30T12:50:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/19' 2023-10-30T12:50:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/15' 2023-10-30T12:50:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/31' 2023-10-30T12:50:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/36' 2023-10-30T12:50:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/20' 2023-10-30T12:50:23.184 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/12' 2023-10-30T12:50:23.184 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/39' 2023-10-30T12:50:23.184 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/10' 2023-10-30T12:50:23.184 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/34' 2023-10-30T12:50:23.184 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/38' 2023-10-30T12:50:23.185 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/11' 2023-10-30T12:50:23.185 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/16' 2023-10-30T12:50:23.185 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/32' 2023-10-30T12:50:23.185 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/30' 2023-10-30T12:50:23.185 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/14' 2023-10-30T12:50:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/18' 2023-10-30T12:50:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/13' 2023-10-30T12:50:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/33' 2023-10-30T12:50:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/17' 2023-10-30T12:50:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/35' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed './165/37' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './165' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/19' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/15' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/31' 2023-10-30T12:50:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/36' 2023-10-30T12:50:23.188 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/20' 2023-10-30T12:50:23.188 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/12' 2023-10-30T12:50:23.188 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/39' 2023-10-30T12:50:23.188 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/10' 2023-10-30T12:50:23.188 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/34' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/38' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/11' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/16' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/32' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/30' 2023-10-30T12:50:23.189 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/14' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/18' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/13' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/33' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/17' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/35' 2023-10-30T12:50:23.190 INFO:tasks.workunit.client.0.smithi008.stdout:removed './166/37' 2023-10-30T12:50:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './166' 2023-10-30T12:50:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/19' 2023-10-30T12:50:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/15' 2023-10-30T12:50:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/31' 2023-10-30T12:50:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/36' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/20' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/12' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/39' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/10' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/34' 2023-10-30T12:50:23.192 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/38' 2023-10-30T12:50:26.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/11' 2023-10-30T12:50:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/16' 2023-10-30T12:50:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/32' 2023-10-30T12:50:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/30' 2023-10-30T12:50:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/14' 2023-10-30T12:50:26.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/18' 2023-10-30T12:50:26.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/13' 2023-10-30T12:50:26.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/33' 2023-10-30T12:50:26.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/17' 2023-10-30T12:50:26.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/35' 2023-10-30T12:50:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './167/37' 2023-10-30T12:50:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './167' 2023-10-30T12:50:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/19' 2023-10-30T12:50:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/15' 2023-10-30T12:50:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/31' 2023-10-30T12:50:26.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/36' 2023-10-30T12:50:26.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/20' 2023-10-30T12:50:26.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/12' 2023-10-30T12:50:26.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/39' 2023-10-30T12:50:26.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/10' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/34' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/38' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/11' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/16' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/32' 2023-10-30T12:50:26.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/30' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/14' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/18' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/13' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/33' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/17' 2023-10-30T12:50:26.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/35' 2023-10-30T12:50:26.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './168/37' 2023-10-30T12:50:26.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './168' 2023-10-30T12:50:26.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/19' 2023-10-30T12:50:26.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/15' 2023-10-30T12:50:26.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/31' 2023-10-30T12:50:26.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/36' 2023-10-30T12:50:26.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/20' 2023-10-30T12:50:26.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/12' 2023-10-30T12:50:26.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/39' 2023-10-30T12:50:26.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/10' 2023-10-30T12:50:26.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/34' 2023-10-30T12:50:26.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/38' 2023-10-30T12:50:26.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/11' 2023-10-30T12:50:26.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/16' 2023-10-30T12:50:26.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/32' 2023-10-30T12:50:26.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/30' 2023-10-30T12:50:26.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/14' 2023-10-30T12:50:26.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/18' 2023-10-30T12:50:26.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/13' 2023-10-30T12:50:26.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/33' 2023-10-30T12:50:26.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/17' 2023-10-30T12:50:26.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/35' 2023-10-30T12:50:26.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './169/37' 2023-10-30T12:50:26.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './169' 2023-10-30T12:50:26.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/19' 2023-10-30T12:50:26.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/15' 2023-10-30T12:50:26.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/31' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/36' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/20' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/12' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/39' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/10' 2023-10-30T12:50:26.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/34' 2023-10-30T12:50:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/38' 2023-10-30T12:50:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/11' 2023-10-30T12:50:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/16' 2023-10-30T12:50:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/32' 2023-10-30T12:50:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/30' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/14' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/18' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/13' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/33' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/17' 2023-10-30T12:50:26.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/35' 2023-10-30T12:50:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './170/37' 2023-10-30T12:50:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './170' 2023-10-30T12:50:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/19' 2023-10-30T12:50:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/15' 2023-10-30T12:50:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/31' 2023-10-30T12:50:26.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/36' 2023-10-30T12:50:26.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/20' 2023-10-30T12:50:26.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/12' 2023-10-30T12:50:26.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/39' 2023-10-30T12:50:26.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/10' 2023-10-30T12:50:26.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/34' 2023-10-30T12:50:26.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/38' 2023-10-30T12:50:26.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/11' 2023-10-30T12:50:26.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/16' 2023-10-30T12:50:26.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/32' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/30' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/14' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/18' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/13' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/33' 2023-10-30T12:50:26.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/17' 2023-10-30T12:50:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/35' 2023-10-30T12:50:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './171/37' 2023-10-30T12:50:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './171' 2023-10-30T12:50:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/19' 2023-10-30T12:50:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/15' 2023-10-30T12:50:26.555 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/31' 2023-10-30T12:50:26.555 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/36' 2023-10-30T12:50:26.555 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/20' 2023-10-30T12:50:26.555 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/12' 2023-10-30T12:50:26.555 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/39' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/10' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/34' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/38' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/11' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/16' 2023-10-30T12:50:26.556 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/32' 2023-10-30T12:50:26.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/30' 2023-10-30T12:50:26.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/14' 2023-10-30T12:50:26.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/18' 2023-10-30T12:50:26.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/13' 2023-10-30T12:50:26.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/33' 2023-10-30T12:50:26.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/17' 2023-10-30T12:50:26.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/35' 2023-10-30T12:50:26.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './172/37' 2023-10-30T12:50:26.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './172' 2023-10-30T12:50:26.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/19' 2023-10-30T12:50:26.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/15' 2023-10-30T12:50:26.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/31' 2023-10-30T12:50:26.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/36' 2023-10-30T12:50:26.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/20' 2023-10-30T12:50:26.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/12' 2023-10-30T12:50:26.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/39' 2023-10-30T12:50:26.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/10' 2023-10-30T12:50:26.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/34' 2023-10-30T12:50:26.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/38' 2023-10-30T12:50:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/11' 2023-10-30T12:50:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/16' 2023-10-30T12:50:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/32' 2023-10-30T12:50:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/30' 2023-10-30T12:50:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/14' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/18' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/13' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/33' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/17' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/35' 2023-10-30T12:50:26.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './173/37' 2023-10-30T12:50:26.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './173' 2023-10-30T12:50:26.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/19' 2023-10-30T12:50:26.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/15' 2023-10-30T12:50:26.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/31' 2023-10-30T12:50:26.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/36' 2023-10-30T12:50:26.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/20' 2023-10-30T12:50:26.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/12' 2023-10-30T12:50:26.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/39' 2023-10-30T12:50:26.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/10' 2023-10-30T12:50:26.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/34' 2023-10-30T12:50:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/38' 2023-10-30T12:50:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/11' 2023-10-30T12:50:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/16' 2023-10-30T12:50:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/32' 2023-10-30T12:50:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/30' 2023-10-30T12:50:26.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/14' 2023-10-30T12:50:26.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/18' 2023-10-30T12:50:26.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/13' 2023-10-30T12:50:26.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/33' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/17' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/35' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './174/37' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './174' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/19' 2023-10-30T12:50:26.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/15' 2023-10-30T12:50:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/31' 2023-10-30T12:50:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/36' 2023-10-30T12:50:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/20' 2023-10-30T12:50:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/12' 2023-10-30T12:50:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/39' 2023-10-30T12:50:26.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/10' 2023-10-30T12:50:26.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/34' 2023-10-30T12:50:26.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/38' 2023-10-30T12:50:26.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/11' 2023-10-30T12:50:26.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/16' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/32' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/30' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/14' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/18' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/13' 2023-10-30T12:50:26.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/33' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/17' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/35' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './175/37' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './175' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/19' 2023-10-30T12:50:26.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/15' 2023-10-30T12:50:26.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/31' 2023-10-30T12:50:26.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/36' 2023-10-30T12:50:26.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/20' 2023-10-30T12:50:26.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/12' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/39' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/10' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/34' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/38' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/11' 2023-10-30T12:50:26.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/16' 2023-10-30T12:50:26.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/32' 2023-10-30T12:50:26.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/30' 2023-10-30T12:50:26.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/14' 2023-10-30T12:50:26.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/18' 2023-10-30T12:50:26.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/13' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/33' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/17' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/35' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './176/37' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './176' 2023-10-30T12:50:26.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/19' 2023-10-30T12:50:26.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/15' 2023-10-30T12:50:29.119 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/31' 2023-10-30T12:50:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/36' 2023-10-30T12:50:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/20' 2023-10-30T12:50:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/12' 2023-10-30T12:50:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/39' 2023-10-30T12:50:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/10' 2023-10-30T12:50:29.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/34' 2023-10-30T12:50:29.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/38' 2023-10-30T12:50:29.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/11' 2023-10-30T12:50:29.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/16' 2023-10-30T12:50:29.121 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/32' 2023-10-30T12:50:29.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/30' 2023-10-30T12:50:29.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/14' 2023-10-30T12:50:29.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/18' 2023-10-30T12:50:29.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/13' 2023-10-30T12:50:29.122 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/33' 2023-10-30T12:50:29.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/17' 2023-10-30T12:50:29.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/35' 2023-10-30T12:50:29.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './177/37' 2023-10-30T12:50:29.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './177' 2023-10-30T12:50:29.123 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/19' 2023-10-30T12:50:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/15' 2023-10-30T12:50:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/31' 2023-10-30T12:50:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/36' 2023-10-30T12:50:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/20' 2023-10-30T12:50:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/12' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/39' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/10' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/34' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/38' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/11' 2023-10-30T12:50:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/16' 2023-10-30T12:50:29.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/32' 2023-10-30T12:50:29.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/30' 2023-10-30T12:50:29.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/14' 2023-10-30T12:50:29.126 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/18' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/13' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/33' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/17' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/35' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed './178/37' 2023-10-30T12:50:29.127 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './178' 2023-10-30T12:50:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/19' 2023-10-30T12:50:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/15' 2023-10-30T12:50:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/31' 2023-10-30T12:50:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/36' 2023-10-30T12:50:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/20' 2023-10-30T12:50:29.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/12' 2023-10-30T12:50:29.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/39' 2023-10-30T12:50:29.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/10' 2023-10-30T12:50:29.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/34' 2023-10-30T12:50:29.129 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/38' 2023-10-30T12:50:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/11' 2023-10-30T12:50:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/16' 2023-10-30T12:50:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/32' 2023-10-30T12:50:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/30' 2023-10-30T12:50:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/14' 2023-10-30T12:50:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/18' 2023-10-30T12:50:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/13' 2023-10-30T12:50:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/33' 2023-10-30T12:50:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/17' 2023-10-30T12:50:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/35' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed './179/37' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './179' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/19' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/15' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/31' 2023-10-30T12:50:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/36' 2023-10-30T12:50:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/20' 2023-10-30T12:50:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/12' 2023-10-30T12:50:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/39' 2023-10-30T12:50:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/10' 2023-10-30T12:50:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/34' 2023-10-30T12:50:29.134 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/38' 2023-10-30T12:50:29.134 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/11' 2023-10-30T12:50:29.134 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/16' 2023-10-30T12:50:29.135 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/32' 2023-10-30T12:50:29.135 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/30' 2023-10-30T12:50:29.136 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/14' 2023-10-30T12:50:29.136 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/18' 2023-10-30T12:50:29.136 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/13' 2023-10-30T12:50:29.137 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/33' 2023-10-30T12:50:29.137 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/17' 2023-10-30T12:50:29.137 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/35' 2023-10-30T12:50:29.137 INFO:tasks.workunit.client.0.smithi008.stdout:removed './180/37' 2023-10-30T12:50:29.138 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './180' 2023-10-30T12:50:29.138 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/19' 2023-10-30T12:50:29.139 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/15' 2023-10-30T12:50:29.139 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/31' 2023-10-30T12:50:29.139 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/36' 2023-10-30T12:50:29.140 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/20' 2023-10-30T12:50:29.140 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/12' 2023-10-30T12:50:29.140 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/39' 2023-10-30T12:50:29.141 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/10' 2023-10-30T12:50:29.141 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/34' 2023-10-30T12:50:29.141 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/38' 2023-10-30T12:50:29.141 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/11' 2023-10-30T12:50:29.141 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/16' 2023-10-30T12:50:29.142 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/32' 2023-10-30T12:50:29.142 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/30' 2023-10-30T12:50:29.142 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/14' 2023-10-30T12:50:29.142 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/18' 2023-10-30T12:50:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/13' 2023-10-30T12:50:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/33' 2023-10-30T12:50:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/17' 2023-10-30T12:50:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/35' 2023-10-30T12:50:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:removed './181/37' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './181' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/19' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/15' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/31' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/36' 2023-10-30T12:50:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/20' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/12' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/39' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/10' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/34' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/38' 2023-10-30T12:50:29.145 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/11' 2023-10-30T12:50:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/16' 2023-10-30T12:50:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/32' 2023-10-30T12:50:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/30' 2023-10-30T12:50:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/14' 2023-10-30T12:50:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/18' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/13' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/33' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/17' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/35' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './182/37' 2023-10-30T12:50:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './182' 2023-10-30T12:50:29.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/19' 2023-10-30T12:50:29.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/15' 2023-10-30T12:50:29.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/31' 2023-10-30T12:50:29.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/36' 2023-10-30T12:50:29.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/20' 2023-10-30T12:50:29.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/12' 2023-10-30T12:50:29.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/39' 2023-10-30T12:50:29.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/10' 2023-10-30T12:50:29.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/34' 2023-10-30T12:50:29.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/38' 2023-10-30T12:50:29.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/11' 2023-10-30T12:50:29.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/16' 2023-10-30T12:50:29.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/32' 2023-10-30T12:50:29.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/30' 2023-10-30T12:50:29.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/14' 2023-10-30T12:50:29.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/18' 2023-10-30T12:50:29.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/13' 2023-10-30T12:50:29.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/33' 2023-10-30T12:50:29.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/17' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/35' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './183/37' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './183' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/19' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/15' 2023-10-30T12:50:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/31' 2023-10-30T12:50:29.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/36' 2023-10-30T12:50:29.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/20' 2023-10-30T12:50:29.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/12' 2023-10-30T12:50:29.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/39' 2023-10-30T12:50:29.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/10' 2023-10-30T12:50:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/34' 2023-10-30T12:50:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/38' 2023-10-30T12:50:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/11' 2023-10-30T12:50:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/16' 2023-10-30T12:50:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/32' 2023-10-30T12:50:29.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/30' 2023-10-30T12:50:29.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/14' 2023-10-30T12:50:29.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/18' 2023-10-30T12:50:29.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/13' 2023-10-30T12:50:29.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/33' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/17' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/35' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './184/37' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './184' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/19' 2023-10-30T12:50:29.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/15' 2023-10-30T12:50:29.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/31' 2023-10-30T12:50:29.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/36' 2023-10-30T12:50:29.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/20' 2023-10-30T12:50:29.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/12' 2023-10-30T12:50:29.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/39' 2023-10-30T12:50:29.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/10' 2023-10-30T12:50:29.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/34' 2023-10-30T12:50:29.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/38' 2023-10-30T12:50:29.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/11' 2023-10-30T12:50:29.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/16' 2023-10-30T12:50:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/32' 2023-10-30T12:50:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/30' 2023-10-30T12:50:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/14' 2023-10-30T12:50:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/18' 2023-10-30T12:50:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/13' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/33' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/17' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/35' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './185/37' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './185' 2023-10-30T12:50:29.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/19' 2023-10-30T12:50:29.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/15' 2023-10-30T12:50:29.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/31' 2023-10-30T12:50:29.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/36' 2023-10-30T12:50:29.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/20' 2023-10-30T12:50:29.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/12' 2023-10-30T12:50:29.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/39' 2023-10-30T12:50:29.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/10' 2023-10-30T12:50:29.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/34' 2023-10-30T12:50:29.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/38' 2023-10-30T12:50:29.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/11' 2023-10-30T12:50:29.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/16' 2023-10-30T12:50:29.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/32' 2023-10-30T12:50:29.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/30' 2023-10-30T12:50:29.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/14' 2023-10-30T12:50:29.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/18' 2023-10-30T12:50:35.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/13' 2023-10-30T12:50:35.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/33' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/17' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/35' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './186/37' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './186' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/19' 2023-10-30T12:50:35.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/15' 2023-10-30T12:50:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/31' 2023-10-30T12:50:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/36' 2023-10-30T12:50:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/20' 2023-10-30T12:50:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/12' 2023-10-30T12:50:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/39' 2023-10-30T12:50:35.585 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/10' 2023-10-30T12:50:35.585 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/34' 2023-10-30T12:50:35.585 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/38' 2023-10-30T12:50:35.585 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/11' 2023-10-30T12:50:35.585 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/16' 2023-10-30T12:50:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/32' 2023-10-30T12:50:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/30' 2023-10-30T12:50:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/14' 2023-10-30T12:50:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/18' 2023-10-30T12:50:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/13' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/33' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/17' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/35' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed './187/37' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './187' 2023-10-30T12:50:35.587 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/19' 2023-10-30T12:50:35.588 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/15' 2023-10-30T12:50:35.588 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/31' 2023-10-30T12:50:35.588 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/36' 2023-10-30T12:50:35.588 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/20' 2023-10-30T12:50:35.588 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/12' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/39' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/10' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/34' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/38' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/11' 2023-10-30T12:50:35.589 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/16' 2023-10-30T12:50:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/32' 2023-10-30T12:50:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/30' 2023-10-30T12:50:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/14' 2023-10-30T12:50:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/18' 2023-10-30T12:50:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/13' 2023-10-30T12:50:35.591 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/33' 2023-10-30T12:50:35.591 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/17' 2023-10-30T12:50:35.591 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/35' 2023-10-30T12:50:35.591 INFO:tasks.workunit.client.0.smithi008.stdout:removed './188/37' 2023-10-30T12:50:35.591 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './188' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/19' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/15' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/31' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/36' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/20' 2023-10-30T12:50:35.592 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/12' 2023-10-30T12:50:35.593 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/39' 2023-10-30T12:50:35.593 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/10' 2023-10-30T12:50:35.593 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/34' 2023-10-30T12:50:35.593 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/38' 2023-10-30T12:50:35.593 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/11' 2023-10-30T12:50:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/16' 2023-10-30T12:50:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/32' 2023-10-30T12:50:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/30' 2023-10-30T12:50:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/14' 2023-10-30T12:50:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/18' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/13' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/33' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/17' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/35' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed './189/37' 2023-10-30T12:50:35.595 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './189' 2023-10-30T12:50:35.596 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/19' 2023-10-30T12:50:35.596 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/15' 2023-10-30T12:50:35.596 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/31' 2023-10-30T12:50:35.596 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/36' 2023-10-30T12:50:35.596 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/20' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/12' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/39' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/10' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/34' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/38' 2023-10-30T12:50:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/11' 2023-10-30T12:50:35.598 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/16' 2023-10-30T12:50:35.598 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/32' 2023-10-30T12:50:35.598 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/30' 2023-10-30T12:50:35.598 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/14' 2023-10-30T12:50:35.598 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/18' 2023-10-30T12:50:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/13' 2023-10-30T12:50:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/33' 2023-10-30T12:50:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/17' 2023-10-30T12:50:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/35' 2023-10-30T12:50:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:removed './190/37' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './190' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/19' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/15' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/31' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/36' 2023-10-30T12:50:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/20' 2023-10-30T12:50:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/12' 2023-10-30T12:50:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/39' 2023-10-30T12:50:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/10' 2023-10-30T12:50:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/34' 2023-10-30T12:50:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/38' 2023-10-30T12:50:35.602 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/11' 2023-10-30T12:50:35.602 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/16' 2023-10-30T12:50:35.602 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/32' 2023-10-30T12:50:35.602 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/30' 2023-10-30T12:50:35.602 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/14' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/18' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/13' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/33' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/17' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/35' 2023-10-30T12:50:35.603 INFO:tasks.workunit.client.0.smithi008.stdout:removed './191/37' 2023-10-30T12:50:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './191' 2023-10-30T12:50:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/19' 2023-10-30T12:50:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/15' 2023-10-30T12:50:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/31' 2023-10-30T12:50:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/36' 2023-10-30T12:50:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/20' 2023-10-30T12:50:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/12' 2023-10-30T12:50:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/39' 2023-10-30T12:50:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/10' 2023-10-30T12:50:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/34' 2023-10-30T12:50:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/38' 2023-10-30T12:50:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/11' 2023-10-30T12:50:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/16' 2023-10-30T12:50:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/32' 2023-10-30T12:50:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/30' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/14' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/18' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/13' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/33' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/17' 2023-10-30T12:50:35.607 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/35' 2023-10-30T12:50:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:removed './192/37' 2023-10-30T12:50:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './192' 2023-10-30T12:50:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/19' 2023-10-30T12:50:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/15' 2023-10-30T12:50:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/31' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/36' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/20' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/12' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/39' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/10' 2023-10-30T12:50:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/34' 2023-10-30T12:50:35.610 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/38' 2023-10-30T12:50:35.610 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/11' 2023-10-30T12:50:35.610 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/16' 2023-10-30T12:50:35.610 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/32' 2023-10-30T12:50:35.610 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/30' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/14' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/18' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/13' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/33' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/17' 2023-10-30T12:50:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/35' 2023-10-30T12:50:35.612 INFO:tasks.workunit.client.0.smithi008.stdout:removed './193/37' 2023-10-30T12:50:35.612 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './193' 2023-10-30T12:50:35.612 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/19' 2023-10-30T12:50:35.612 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/15' 2023-10-30T12:50:35.612 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/31' 2023-10-30T12:50:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/36' 2023-10-30T12:50:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/20' 2023-10-30T12:50:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/12' 2023-10-30T12:50:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/39' 2023-10-30T12:50:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/10' 2023-10-30T12:50:35.614 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/34' 2023-10-30T12:50:35.614 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/38' 2023-10-30T12:50:35.614 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/11' 2023-10-30T12:50:35.614 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/16' 2023-10-30T12:50:35.614 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/32' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/30' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/14' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/18' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/13' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/33' 2023-10-30T12:50:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/17' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/35' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed './194/37' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './194' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/19' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/15' 2023-10-30T12:50:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/31' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/36' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/20' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/12' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/39' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/10' 2023-10-30T12:50:35.617 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/34' 2023-10-30T12:50:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/38' 2023-10-30T12:50:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/11' 2023-10-30T12:50:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/16' 2023-10-30T12:50:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/32' 2023-10-30T12:50:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/30' 2023-10-30T12:50:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/14' 2023-10-30T12:50:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/18' 2023-10-30T12:50:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/13' 2023-10-30T12:50:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/33' 2023-10-30T12:50:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/17' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/35' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed './195/37' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './195' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/19' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/15' 2023-10-30T12:50:35.620 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/31' 2023-10-30T12:50:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/36' 2023-10-30T12:50:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/20' 2023-10-30T12:50:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/12' 2023-10-30T12:50:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/39' 2023-10-30T12:50:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/10' 2023-10-30T12:50:37.231 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:37.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.382+0000 7f9c08a9c640 1 Processor -- start 2023-10-30T12:50:37.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.382+0000 7f9c08a9c640 1 -- start start 2023-10-30T12:50:37.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.382+0000 7f9c08a9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c0406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c08a9c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 0x7f9c040709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c08a9c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040c2310 con 0x7f9c040cb7d0 2023-10-30T12:50:37.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c08a9c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c04156710 con 0x7f9c04070520 2023-10-30T12:50:37.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c08a9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040c4360 con 0x7f9c04150830 2023-10-30T12:50:37.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9bfbfff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f9c040cb7d0 0x7f9c040c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:59774/0 (socket says 172.21.15.8:59774) 2023-10-30T12:50:37.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9bfbfff640 1 -- 172.21.15.8:0/460998453 learned_addr learned my addr 172.21.15.8:0/460998453 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:37.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c02d9d640 1 --2- 172.21.15.8:0/460998453 >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 0x7f9c040709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c0259c640 1 --2- 172.21.15.8:0/460998453 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c0406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162434047 0 0) 0x7f9c040c2310 con 0x7f9c040cb7d0 2023-10-30T12:50:37.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003530 con 0x7f9c040cb7d0 2023-10-30T12:50:37.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c0259c640 1 -- 172.21.15.8:0/460998453 >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 msgr2=0x7f9c040709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:37.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c0259c640 1 --2- 172.21.15.8:0/460998453 >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 0x7f9c040709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 693534020 0 0) 0x7f9bf0003530 con 0x7f9c040cb7d0 2023-10-30T12:50:37.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c0259c640 1 -- 172.21.15.8:0/460998453 >> v1:172.21.15.8:6789/0 conn(0x7f9c040cb7d0 legacy=0x7f9c040c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c0259c640 1 -- 172.21.15.8:0/460998453 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c041569b0 con 0x7f9c04150830 2023-10-30T12:50:37.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.386+0000 7f9c02d9d640 1 --2- 172.21.15.8:0/460998453 >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 0x7f9c040709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:50:37.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c0259c640 1 --2- 172.21.15.8:0/460998453 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c0406b500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf177f98e320c05 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.395 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec001680 con 0x7f9c04150830 2023-10-30T12:50:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40a85d0 con 0x7f9c04150830 2023-10-30T12:50:37.396 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c01d9b640 1 -- 172.21.15.8:0/460998453 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40a8960 con 0x7f9c04150830 2023-10-30T12:50:37.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- 172.21.15.8:0/460998453 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 msgr2=0x7f9c0406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/460998453 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c0406b500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401eb90 comp rx=0 tx=0).stop 2023-10-30T12:50:37.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- 172.21.15.8:0/460998453 shutdown_connections 2023-10-30T12:50:37.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/460998453 >> v2:172.21.15.46:3300/0 conn(0x7f9c04070520 0x7f9c040709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/460998453 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c0406b500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- 172.21.15.8:0/460998453 >> 172.21.15.8:0/460998453 conn(0x7f9c040cac70 msgr2=0x7f9c040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:37.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- 172.21.15.8:0/460998453 shutdown_connections 2023-10-30T12:50:37.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- 172.21.15.8:0/460998453 wait complete. 2023-10-30T12:50:37.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 Processor -- start 2023-10-30T12:50:37.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.390+0000 7f9c08a9c640 1 -- start start 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c040cb7d0 0x7f9c04148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c04148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415aa50 con 0x7f9c04070520 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04158df0 con 0x7f9c040cb7d0 2023-10-30T12:50:37.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04158ac0 con 0x7f9c04150830 2023-10-30T12:50:37.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58628/0 (socket says 172.21.15.8:58628) 2023-10-30T12:50:37.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 -- 172.21.15.8:0/1835822709 learned_addr learned my addr 172.21.15.8:0/1835822709 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:37.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9bfbfff640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c040cb7d0 0x7f9c04148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c02d9d640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c04148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 -- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 msgr2=0x7f9c04148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c04148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 -- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c040cb7d0 msgr2=0x7f9c04148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c040cb7d0 0x7f9c04148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 -- 172.21.15.8:0/1835822709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc009a70 con 0x7f9c04070520 2023-10-30T12:50:37.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c0259c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40b0850 con 0x7f9c04070520 2023-10-30T12:50:37.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40b09d0 con 0x7f9c04070520 2023-10-30T12:50:37.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf4009420 con 0x7f9c04070520 2023-10-30T12:50:37.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40b0d80 con 0x7f9c04070520 2023-10-30T12:50:37.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c041561d0 con 0x7f9c04070520 2023-10-30T12:50:37.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c04066510 con 0x7f9c04070520 2023-10-30T12:50:37.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.394+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9bf40b7040 con 0x7f9c04070520 2023-10-30T12:50:37.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.398+0000 7f9bfaffd640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 0x7f9be4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.398+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f9bf4121170 con 0x7f9c04070520 2023-10-30T12:50:37.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.398+0000 7f9bfbfff640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 0x7f9be4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.398+0000 7f9bfbfff640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 0x7f9be4068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c041499a0 tx=0x7f9bec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.402+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9bf40ee200 con 0x7f9c04070520 2023-10-30T12:50:37.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.602+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c0404e170 con 0x7f9c04070520 2023-10-30T12:50:37.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.602+0000 7f9bfaffd640 1 -- 172.21.15.8:0/1835822709 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f9bf40f1820 con 0x7f9c04070520 2023-10-30T12:50:37.610 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:37.610 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:37.611 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:37.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 msgr2=0x7f9be4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 0x7f9be4068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c041499a0 tx=0x7f9bec009040 comp rx=0 tx=0).stop 2023-10-30T12:50:37.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 msgr2=0x7f9c0414b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401b930 comp rx=0 tx=0).stop 2023-10-30T12:50:37.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 shutdown_connections 2023-10-30T12:50:37.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9be40664d0 0x7f9be4068990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c040cb7d0 0x7f9c04148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9c04150830 0x7f9c04148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 --2- 172.21.15.8:0/1835822709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c04070520 0x7f9c0414b7c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 >> 172.21.15.8:0/1835822709 conn(0x7f9c040cac70 msgr2=0x7f9c040c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:37.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 shutdown_connections 2023-10-30T12:50:37.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.606+0000 7f9c08a9c640 1 -- 172.21.15.8:0/1835822709 wait complete. 2023-10-30T12:50:37.633 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2023-10-30T12:50:37.633 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-30T12:50:37.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:50:37.649 INFO:tasks.ceph.mds.f:Stopped 2023-10-30T12:50:37.649 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 Processor -- start 2023-10-30T12:50:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 -- start start 2023-10-30T12:50:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2f0815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080c2aa0 con 0x7f2f080701a0 2023-10-30T12:50:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080708e0 con 0x7f2f081537f0 2023-10-30T12:50:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0fa6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c3ab0 con 0x7f2f0806ba00 2023-10-30T12:50:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0d006640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2f080701a0 0x7f2f0806b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:59790/0 (socket says 172.21.15.8:59790) 2023-10-30T12:50:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0d006640 1 -- 172.21.15.8:0/1693210263 learned_addr learned my addr 172.21.15.8:0/1693210263 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.786+0000 7f2f0e008640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2f0815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400546103 0 0) 0x7f2f080c2aa0 con 0x7f2f080701a0 2023-10-30T12:50:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ef4003530 con 0x7f2f080701a0 2023-10-30T12:50:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731066728 0 0) 0x7f2f080708e0 con 0x7f2f081537f0 2023-10-30T12:50:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f080c2aa0 con 0x7f2f081537f0 2023-10-30T12:50:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881127213 0 0) 0x7f2ef4003530 con 0x7f2f080701a0 2023-10-30T12:50:37.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 msgr2=0x7f2f0815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:37.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2f0815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 >> v1:172.21.15.46:6789/0 conn(0x7f2f081537f0 legacy=0x7f2f080ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080c1eb0 con 0x7f2f080701a0 2023-10-30T12:50:37.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f2ef8001680 con 0x7f2f080701a0 2023-10-30T12:50:37.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 >> v1:172.21.15.8:6789/0 conn(0x7f2f080701a0 legacy=0x7f2f0806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef4003b40 0x7f2ef4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef40047d0 0x7f2ef4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2ef4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef400ba40 con 0x7f2ef40047d0 2023-10-30T12:50:37.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef400bb80 con 0x7f2ef4003b40 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef400bd10 con 0x7f2f0806ba00 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef40047d0 0x7f2ef4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d807640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef4003b40 0x7f2ef4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0e008640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2ef4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 -- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 msgr2=0x7f2ef4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:37.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f0806ba00 0x7f2ef4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 -- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef4003b40 msgr2=0x7f2ef4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:37.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef4003b40 0x7f2ef4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.790+0000 7f2f0d006640 1 -- 172.21.15.8:0/1693210263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef4003530 con 0x7f2ef40047d0 2023-10-30T12:50:37.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef40047d0 0x7f2ef4004c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c160 tx=0x7f2ef801f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1143679a7b82e61 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80a8ce0 con 0x7f2ef40047d0 2023-10-30T12:50:37.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80a8e60 con 0x7f2ef40047d0 2023-10-30T12:50:37.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0c805640 1 -- 172.21.15.8:0/1693210263 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8005b20 con 0x7f2ef40047d0 2023-10-30T12:50:37.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1693210263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef40047d0 msgr2=0x7f2ef4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1693210263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef40047d0 0x7f2ef4004c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c160 tx=0x7f2ef801f2b0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0e008640 1 -- 172.21.15.8:0/1693210263 reap_dead start 2023-10-30T12:50:37.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1693210263 shutdown_connections 2023-10-30T12:50:37.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1693210263 >> 172.21.15.8:0/1693210263 conn(0x7f2f08071fb0 msgr2=0x7f2f080723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:37.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1693210263 shutdown_connections 2023-10-30T12:50:37.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1693210263 wait complete. 2023-10-30T12:50:37.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 Processor -- start 2023-10-30T12:50:37.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- start start 2023-10-30T12:50:37.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 0x7f2f080c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f08147b40 0x7f2f08147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080725c0 con 0x7f2f08154150 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0814f460 con 0x7f2ef40047d0 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0fa6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08156d30 con 0x7f2f08147b40 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.794+0000 7f2f0d006640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58640/0 (socket says 172.21.15.8:58640) 2023-10-30T12:50:37.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 -- 172.21.15.8:0/1636379818 learned_addr learned my addr 172.21.15.8:0/1636379818 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:37.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d807640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 0x7f2f080c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0e008640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f08147b40 0x7f2f08147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 -- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f08147b40 msgr2=0x7f2f08147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f08147b40 0x7f2f08147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 -- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 msgr2=0x7f2f080c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:37.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 0x7f2f080c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:37.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 -- 172.21.15.8:0/1636379818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080ca2d0 con 0x7f2f08154150 2023-10-30T12:50:37.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d807640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 0x7f2f080c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0d006640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ae040 tx=0x7f2ef4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80049f0 con 0x7f2f08154150 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80b6070 con 0x7f2f08154150 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0814f690 con 0x7f2f08154150 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8004d90 con 0x7f2f08154150 2023-10-30T12:50:37.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f080c8070 con 0x7f2f08154150 2023-10-30T12:50:37.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0804e170 con 0x7f2f08154150 2023-10-30T12:50:37.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.798+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ef80070d0 con 0x7f2f08154150 2023-10-30T12:50:37.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.802+0000 7f2efe7fc640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 0x7f2eec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:37.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.802+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f2ef80b8070 con 0x7f2f08154150 2023-10-30T12:50:37.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.802+0000 7f2f0d807640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 0x7f2eec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:37.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.802+0000 7f2f0d807640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 0x7f2eec068990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007fd0 tx=0x7f2f04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:37.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.806+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ef80ee650 con 0x7f2f08154150 2023-10-30T12:50:38.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.994+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f080c2420 con 0x7f2f08154150 2023-10-30T12:50:38.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.994+0000 7f2efe7fc640 1 -- 172.21.15.8:0/1636379818 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f2ef80f1c70 con 0x7f2f08154150 2023-10-30T12:50:38.001 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:38.002 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:38.002 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 msgr2=0x7f2eec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 0x7f2eec068990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007fd0 tx=0x7f2f04009040 comp rx=0 tx=0).stop 2023-10-30T12:50:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 msgr2=0x7f2f0814e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ae040 tx=0x7f2ef4005800 comp rx=0 tx=0).stop 2023-10-30T12:50:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 shutdown_connections 2023-10-30T12:50:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2eec0664d0 0x7f2eec068990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2f08147b40 0x7f2f08147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f08154150 0x7f2f0814e950 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 --2- 172.21.15.8:0/1636379818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ef40047d0 0x7f2f080c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:38.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 >> 172.21.15.8:0/1636379818 conn(0x7f2f08071fb0 msgr2=0x7f2f0814fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:38.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 shutdown_connections 2023-10-30T12:50:38.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:37.998+0000 7f2f0fa6b640 1 -- 172.21.15.8:0/1636379818 wait complete. 2023-10-30T12:50:38.022 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:40.023 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 Processor -- start 2023-10-30T12:50:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 -- start start 2023-10-30T12:50:40.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f65fc151840 0x7f65fc0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc0cb8b0 0x7f65fc06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0c5030 con 0x7f65fc151c20 2023-10-30T12:50:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc157830 con 0x7f65fc151840 2023-10-30T12:50:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f6601c09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc157b10 con 0x7f65fc0cb8b0 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f65fc151840 0x7f65fc0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f65fc151840 0x7f65fc0c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38820/0 (socket says 172.21.15.8:38820) 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 -- 172.21.15.8:0/2010785433 learned_addr learned my addr 172.21.15.8:0/2010785433 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fbfff640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc0cb8b0 0x7f65fc06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 -- 172.21.15.8:0/2010785433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc0cb8b0 msgr2=0x7f65fc06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc0cb8b0 0x7f65fc06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 -- 172.21.15.8:0/2010785433 >> v1:172.21.15.46:6789/0 conn(0x7f65fc151c20 legacy=0x7f65fc072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.174+0000 7f65fb7fe640 1 -- 172.21.15.8:0/2010785433 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc157df0 con 0x7f65fc151840 2023-10-30T12:50:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f65fb7fe640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151840 0x7f65fc0c49a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009870 tx=0x7f65ec01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2893b8a9523288c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f65fa7fc640 1 -- 172.21.15.8:0/2010785433 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0a8a30 con 0x7f65fc151840 2023-10-30T12:50:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f65fa7fc640 1 -- 172.21.15.8:0/2010785433 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec0a8b90 con 0x7f65fc151840 2023-10-30T12:50:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f65fa7fc640 1 -- 172.21.15.8:0/2010785433 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0b0440 con 0x7f65fc151840 2023-10-30T12:50:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 -- 172.21.15.8:0/2010785433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151840 msgr2=0x7f65fc0c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151840 0x7f65fc0c49a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009870 tx=0x7f65ec01ed30 comp rx=0 tx=0).stop 2023-10-30T12:50:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 -- 172.21.15.8:0/2010785433 shutdown_connections 2023-10-30T12:50:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc0cb8b0 0x7f65fc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 --2- 172.21.15.8:0/2010785433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151840 0x7f65fc0c49a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 -- 172.21.15.8:0/2010785433 >> 172.21.15.8:0/2010785433 conn(0x7f65fc0cac70 msgr2=0x7f65fc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 -- 172.21.15.8:0/2010785433 shutdown_connections 2023-10-30T12:50:40.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.178+0000 7f6601c09640 1 -- 172.21.15.8:0/2010785433 wait complete. 2023-10-30T12:50:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 Processor -- start 2023-10-30T12:50:40.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 -- start start 2023-10-30T12:50:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65fc0cb8b0 0x7f65fc0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc151840 0x7f65fc157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1ef8d0 con 0x7f65fc151c20 2023-10-30T12:50:40.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc06c210 con 0x7f65fc0cb8b0 2023-10-30T12:50:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc157590 con 0x7f65fc151840 2023-10-30T12:50:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38824/0 (socket says 172.21.15.8:38824) 2023-10-30T12:50:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 -- 172.21.15.8:0/1503729112 learned_addr learned my addr 172.21.15.8:0/1503729112 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:40.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fb7fe640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65fc0cb8b0 0x7f65fc0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65faffd640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc151840 0x7f65fc157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 -- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc151840 msgr2=0x7f65fc157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc151840 0x7f65fc157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 -- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65fc0cb8b0 msgr2=0x7f65fc0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65fc0cb8b0 0x7f65fc0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 -- 172.21.15.8:0/1503729112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec0094d0 con 0x7f65fc151c20 2023-10-30T12:50:40.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65fbfff640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007e40 tx=0x7f65f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014620 con 0x7f65fc151c20 2023-10-30T12:50:40.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f400ee90 con 0x7f65fc151c20 2023-10-30T12:50:40.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f40149e0 con 0x7f65fc151c20 2023-10-30T12:50:40.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.182+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc1eefc0 con 0x7f65fc151c20 2023-10-30T12:50:40.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc1f32c0 con 0x7f65fc151c20 2023-10-30T12:50:40.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc1efa40 con 0x7f65fc151c20 2023-10-30T12:50:40.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f65f4014b40 con 0x7f65fc151c20 2023-10-30T12:50:40.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f65e3fff640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 0x7f65e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:40.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f65f4084bf0 con 0x7f65fc151c20 2023-10-30T12:50:40.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f65fb7fe640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 0x7f65e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:40.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.186+0000 7f65fb7fe640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 0x7f65e8068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0b2000 tx=0x7f65ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:40.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.190+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f65f4051d10 con 0x7f65fc151c20 2023-10-30T12:50:40.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.378+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65fc04e170 con 0x7f65fc151c20 2023-10-30T12:50:40.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f65e3fff640 1 -- 172.21.15.8:0/1503729112 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f65f4055330 con 0x7f65fc151c20 2023-10-30T12:50:40.387 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:40.388 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:40.389 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:40.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 msgr2=0x7f65e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 0x7f65e8068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0b2000 tx=0x7f65ec008040 comp rx=0 tx=0).stop 2023-10-30T12:50:40.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 msgr2=0x7f65fc146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:40.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007e40 tx=0x7f65f400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 shutdown_connections 2023-10-30T12:50:40.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f65e80664a0 0x7f65e8068960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65fc151c20 0x7f65fc146050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f65fc151840 0x7f65fc157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 --2- 172.21.15.8:0/1503729112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65fc0cb8b0 0x7f65fc0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:40.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 >> 172.21.15.8:0/1503729112 conn(0x7f65fc0cac70 msgr2=0x7f65fc1e1390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:40.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 shutdown_connections 2023-10-30T12:50:40.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:40.382+0000 7f6601c09640 1 -- 172.21.15.8:0/1503729112 wait complete. 2023-10-30T12:50:40.408 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:42.409 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:42.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 Processor -- start 2023-10-30T12:50:42.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 -- start start 2023-10-30T12:50:42.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd638151c20 0x7fd6380c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 0x7fd63806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380c5030 con 0x7fd638151840 2023-10-30T12:50:42.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd638157830 con 0x7fd638151c20 2023-10-30T12:50:42.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63e679640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd638157b10 con 0x7fd6380cb8b0 2023-10-30T12:50:42.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd637fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd638151840 0x7fd638072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:33622/0 (socket says 172.21.15.8:33622) 2023-10-30T12:50:42.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd637fff640 1 -- 172.21.15.8:0/1970189724 learned_addr learned my addr 172.21.15.8:0/1970189724 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:42.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd6377fe640 1 --2- 172.21.15.8:0/1970189724 >> v2:172.21.15.8:3300/0 conn(0x7fd638151c20 0x7fd6380c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.558+0000 7fd63cc16640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 0x7fd63806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd6377fe640 1 -- 172.21.15.8:0/1970189724 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 msgr2=0x7fd63806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:42.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd6377fe640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 0x7fd63806b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd6377fe640 1 -- 172.21.15.8:0/1970189724 >> v1:172.21.15.8:6789/0 conn(0x7fd638151840 legacy=0x7fd638072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd6377fe640 1 -- 172.21.15.8:0/1970189724 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638157df0 con 0x7fd638151c20 2023-10-30T12:50:42.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63cc16640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 0x7fd63806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:42.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd636ffd640 1 -- 172.21.15.8:0/1970189724 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951516998 0 0) 0x7fd6380c5030 con 0x7fd638151840 2023-10-30T12:50:42.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd6377fe640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638151c20 0x7fd6380c49a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0097c0 tx=0x7fd62c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bac5d1f737aca1d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd636ffd640 1 -- 172.21.15.8:0/1970189724 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0a8970 con 0x7fd638151c20 2023-10-30T12:50:42.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd636ffd640 1 -- 172.21.15.8:0/1970189724 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c0a8ad0 con 0x7fd638151c20 2023-10-30T12:50:42.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd636ffd640 1 -- 172.21.15.8:0/1970189724 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0a8e60 con 0x7fd638151c20 2023-10-30T12:50:42.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 -- 172.21.15.8:0/1970189724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638151c20 msgr2=0x7fd6380c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638151c20 0x7fd6380c49a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0097c0 tx=0x7fd62c01ec70 comp rx=0 tx=0).stop 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 -- 172.21.15.8:0/1970189724 shutdown_connections 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd6380cb8b0 0x7fd63806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 --2- 172.21.15.8:0/1970189724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638151c20 0x7fd6380c49a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 -- 172.21.15.8:0/1970189724 >> 172.21.15.8:0/1970189724 conn(0x7fd6380cac70 msgr2=0x7fd6380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:42.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.562+0000 7fd63e679640 1 -- 172.21.15.8:0/1970189724 shutdown_connections 2023-10-30T12:50:42.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 -- 172.21.15.8:0/1970189724 wait complete. 2023-10-30T12:50:42.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 Processor -- start 2023-10-30T12:50:42.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 -- start start 2023-10-30T12:50:42.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd638151840 0x7fd6380c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd638151c20 0x7fd638156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381ef150 con 0x7fd6380cb8b0 2023-10-30T12:50:42.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381ef410 con 0x7fd638151840 2023-10-30T12:50:42.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63e679640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381ef6f0 con 0x7fd638151c20 2023-10-30T12:50:42.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38842/0 (socket says 172.21.15.8:38842) 2023-10-30T12:50:42.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 -- 172.21.15.8:0/4030193403 learned_addr learned my addr 172.21.15.8:0/4030193403 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:42.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd63cc16640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd638151c20 0x7fd638156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd6377fe640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd638151840 0x7fd6380c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 -- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd638151c20 msgr2=0x7fd638156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd638151c20 0x7fd638156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 -- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd638151840 msgr2=0x7fd6380c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd638151840 0x7fd6380c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 -- 172.21.15.8:0/4030193403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009420 con 0x7fd6380cb8b0 2023-10-30T12:50:42.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.566+0000 7fd637fff640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6280011f0 tx=0x7fd62800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280074e0 con 0x7fd6380cb8b0 2023-10-30T12:50:42.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628007640 con 0x7fd6380cb8b0 2023-10-30T12:50:42.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628004a10 con 0x7fd6380cb8b0 2023-10-30T12:50:42.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638155b80 con 0x7fd6380cb8b0 2023-10-30T12:50:42.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638156140 con 0x7fd6380cb8b0 2023-10-30T12:50:42.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63804e170 con 0x7fd6380cb8b0 2023-10-30T12:50:42.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd628004b70 con 0x7fd6380cb8b0 2023-10-30T12:50:42.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 0x7fd624068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:42.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fd6280834a0 con 0x7fd6380cb8b0 2023-10-30T12:50:42.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd6377fe640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 0x7fd624068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:42.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.570+0000 7fd6377fe640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 0x7fd624068990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0b2000 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:42.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.574+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd6280504a0 con 0x7fd6380cb8b0 2023-10-30T12:50:42.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.766+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd638155060 con 0x7fd6380cb8b0 2023-10-30T12:50:42.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd634ff9640 1 -- 172.21.15.8:0/4030193403 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7fd628018090 con 0x7fd6380cb8b0 2023-10-30T12:50:42.775 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:42.775 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:42.777 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:42.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 msgr2=0x7fd624068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 0x7fd624068990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0b2000 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2023-10-30T12:50:42.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 msgr2=0x7fd6381ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:42.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6280011f0 tx=0x7fd62800c9b0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 shutdown_connections 2023-10-30T12:50:42.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fd6240664d0 0x7fd624068990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd638151840 0x7fd6380c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fd638151c20 0x7fd638156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 --2- 172.21.15.8:0/4030193403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380cb8b0 0x7fd6381ee880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:42.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 >> 172.21.15.8:0/4030193403 conn(0x7fd6380cac70 msgr2=0x7fd6380c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:42.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 shutdown_connections 2023-10-30T12:50:42.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:42.770+0000 7fd63e679640 1 -- 172.21.15.8:0/4030193403 wait complete. 2023-10-30T12:50:42.796 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:44.797 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:44.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 Processor -- start 2023-10-30T12:50:44.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 -- start start 2023-10-30T12:50:44.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23a814fc50 0x7f23a806fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f23a806be90 0x7f23a806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a80c55f0 con 0x7f23a80c6320 2023-10-30T12:50:44.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80c5f00 con 0x7f23a806be90 2023-10-30T12:50:44.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23ade9c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a81567b0 con 0x7f23a814fc50 2023-10-30T12:50:44.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23a7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f23a806be90 0x7f23a806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23a7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f23a806be90 0x7f23a806c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38846/0 (socket says 172.21.15.8:38846) 2023-10-30T12:50:44.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23a7fff640 1 -- 172.21.15.8:0/623893862 learned_addr learned my addr 172.21.15.8:0/623893862 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:44.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.946+0000 7f23a77fe640 1 --2- 172.21.15.8:0/623893862 >> v2:172.21.15.46:3300/0 conn(0x7f23a814fc50 0x7f23a806fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a7fff640 1 -- 172.21.15.8:0/623893862 >> v2:172.21.15.46:3300/0 conn(0x7f23a814fc50 msgr2=0x7f23a806fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:44.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a7fff640 1 --2- 172.21.15.8:0/623893862 >> v2:172.21.15.46:3300/0 conn(0x7f23a814fc50 0x7f23a806fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:44.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a7fff640 1 -- 172.21.15.8:0/623893862 >> v1:172.21.15.46:6789/0 conn(0x7f23a80c6320 legacy=0x7f23a8070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:44.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a7fff640 1 -- 172.21.15.8:0/623893862 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a8156a70 con 0x7f23a806be90 2023-10-30T12:50:44.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a7fff640 1 --2- 172.21.15.8:0/623893862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a806c2e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23a806f6e0 tx=0x7f23a001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6feab56974e8995 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:44.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a67fc640 1 -- 172.21.15.8:0/623893862 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00a5780 con 0x7f23a806be90 2023-10-30T12:50:44.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a67fc640 1 -- 172.21.15.8:0/623893862 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a00a58e0 con 0x7f23a806be90 2023-10-30T12:50:44.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23a67fc640 1 -- 172.21.15.8:0/623893862 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00a5c70 con 0x7f23a806be90 2023-10-30T12:50:44.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23ade9c640 1 -- 172.21.15.8:0/623893862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 msgr2=0x7f23a806c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:44.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.950+0000 7f23ade9c640 1 --2- 172.21.15.8:0/623893862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a806c2e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23a806f6e0 tx=0x7f23a001c940 comp rx=0 tx=0).stop 2023-10-30T12:50:44.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- 172.21.15.8:0/623893862 shutdown_connections 2023-10-30T12:50:44.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 --2- 172.21.15.8:0/623893862 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a806c2e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:44.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 --2- 172.21.15.8:0/623893862 >> v2:172.21.15.46:3300/0 conn(0x7f23a814fc50 0x7f23a806fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:44.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- 172.21.15.8:0/623893862 >> 172.21.15.8:0/623893862 conn(0x7f23a80cac70 msgr2=0x7f23a80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:44.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- 172.21.15.8:0/623893862 shutdown_connections 2023-10-30T12:50:44.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- 172.21.15.8:0/623893862 wait complete. 2023-10-30T12:50:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 Processor -- start 2023-10-30T12:50:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- start start 2023-10-30T12:50:44.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23a80c6320 0x7f23a81435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 0x7f23a8143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a81448b0 con 0x7f23a806be90 2023-10-30T12:50:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a8145980 con 0x7f23a814fc50 2023-10-30T12:50:44.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23ade9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80c37f0 con 0x7f23a80c6320 2023-10-30T12:50:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38858/0 (socket says 172.21.15.8:38858) 2023-10-30T12:50:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 -- 172.21.15.8:0/1639350157 learned_addr learned my addr 172.21.15.8:0/1639350157 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a6ffd640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23a80c6320 0x7f23a81435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a7fff640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 0x7f23a8143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 -- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23a80c6320 msgr2=0x7f23a81435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23a80c6320 0x7f23a81435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 -- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 msgr2=0x7f23a8143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 0x7f23a8143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:44.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a77fe640 1 -- 172.21.15.8:0/1639350157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0009b30 con 0x7f23a806be90 2023-10-30T12:50:44.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.954+0000 7f23a7fff640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 0x7f23a8143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:44.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23a77fe640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f239800e8a0 tx=0x7f239800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:44.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398009510 con 0x7f23a806be90 2023-10-30T12:50:44.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239800c940 con 0x7f23a806be90 2023-10-30T12:50:44.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239800ccb0 con 0x7f23a806be90 2023-10-30T12:50:44.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a80c3230 con 0x7f23a806be90 2023-10-30T12:50:44.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a8145e10 con 0x7f23a806be90 2023-10-30T12:50:44.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a804e170 con 0x7f23a806be90 2023-10-30T12:50:44.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2398013470 con 0x7f23a806be90 2023-10-30T12:50:44.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 0x7f2394068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:44.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f2398084210 con 0x7f23a806be90 2023-10-30T12:50:44.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23a6ffd640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 0x7f2394068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:44.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.958+0000 7f23a6ffd640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 0x7f2394068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23a814d570 tx=0x7f239c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:44.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:44.962+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23980513d0 con 0x7f23a806be90 2023-10-30T12:50:45.005 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:45.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:50:45.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.154+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a80c7300 con 0x7f23a806be90 2023-10-30T12:50:45.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.154+0000 7f238ffff640 1 -- 172.21.15.8:0/1639350157 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f23980549f0 con 0x7f23a806be90 2023-10-30T12:50:45.162 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:45.162 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 msgr2=0x7f2394068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 0x7f2394068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23a814d570 tx=0x7f239c009040 comp rx=0 tx=0).stop 2023-10-30T12:50:45.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 msgr2=0x7f23a8145440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:45.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f239800e8a0 tx=0x7f239800eca0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 shutdown_connections 2023-10-30T12:50:45.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23940664a0 0x7f2394068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a814fc50 0x7f23a8143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23a80c6320 0x7f23a81435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 --2- 172.21.15.8:0/1639350157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23a806be90 0x7f23a8145440 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:45.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 >> 172.21.15.8:0/1639350157 conn(0x7f23a80cac70 msgr2=0x7f23a80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:45.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 shutdown_connections 2023-10-30T12:50:45.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:45.158+0000 7f23ade9c640 1 -- 172.21.15.8:0/1639350157 wait complete. 2023-10-30T12:50:45.183 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:47.184 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:47.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.330+0000 7fe204609640 1 Processor -- start 2023-10-30T12:50:47.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.330+0000 7fe204609640 1 -- start start 2023-10-30T12:50:47.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe204609640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc1516c0 0x7fe1fc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe204609640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe1fc0c2500 0x7fe1fc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe204609640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc151aa0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe204609640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c74a0 con 0x7fe1fc0c2500 2023-10-30T12:50:47.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe204609640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c2e90 con 0x7fe1fc1516c0 2023-10-30T12:50:47.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe201ba4640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe1fc0c82d0 0x7fe1fc0ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:33634/0 (socket says 172.21.15.8:33634) 2023-10-30T12:50:47.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe201ba4640 1 -- 172.21.15.8:0/3380785086 learned_addr learned my addr 172.21.15.8:0/3380785086 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:47.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe202ba6640 1 --2- 172.21.15.8:0/3380785086 >> v2:172.21.15.8:3300/0 conn(0x7fe1fc0c2500 0x7fe1fc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe2023a5640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc1516c0 0x7fe1fc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe202ba6640 1 -- 172.21.15.8:0/3380785086 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc1516c0 msgr2=0x7fe1fc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:47.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe202ba6640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc1516c0 0x7fe1fc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe202ba6640 1 -- 172.21.15.8:0/3380785086 >> v1:172.21.15.8:6789/0 conn(0x7fe1fc0c82d0 legacy=0x7fe1fc0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe202ba6640 1 -- 172.21.15.8:0/3380785086 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc158af0 con 0x7fe1fc0c2500 2023-10-30T12:50:47.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.334+0000 7fe2013a3640 1 -- 172.21.15.8:0/3380785086 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165573448 0 0) 0x7fe1fc151aa0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe202ba6640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c2500 0x7fe1fc0c2950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009dc0 tx=0x7fe1f401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29881705a51f25fd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe2013a3640 1 -- 172.21.15.8:0/3380785086 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40a57f0 con 0x7fe1fc0c2500 2023-10-30T12:50:47.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe2013a3640 1 -- 172.21.15.8:0/3380785086 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f40a5950 con 0x7fe1fc0c2500 2023-10-30T12:50:47.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe2013a3640 1 -- 172.21.15.8:0/3380785086 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40a5ce0 con 0x7fe1fc0c2500 2023-10-30T12:50:47.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- 172.21.15.8:0/3380785086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c2500 msgr2=0x7fe1fc0c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c2500 0x7fe1fc0c2950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009dc0 tx=0x7fe1f401c930 comp rx=0 tx=0).stop 2023-10-30T12:50:47.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- 172.21.15.8:0/3380785086 shutdown_connections 2023-10-30T12:50:47.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c2500 0x7fe1fc0c2950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- 172.21.15.8:0/3380785086 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc1516c0 0x7fe1fc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- 172.21.15.8:0/3380785086 >> 172.21.15.8:0/3380785086 conn(0x7fe1fc071fb0 msgr2=0x7fe1fc0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:47.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- 172.21.15.8:0/3380785086 shutdown_connections 2023-10-30T12:50:47.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- 172.21.15.8:0/3380785086 wait complete. 2023-10-30T12:50:47.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 Processor -- start 2023-10-30T12:50:47.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- start start 2023-10-30T12:50:47.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 0x7fe1fc1498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 0x7fe1fc145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc14b360 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc1f23d0 con 0x7fe1fc1516c0 2023-10-30T12:50:47.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.338+0000 7fe204609640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc1f2860 con 0x7fe1fc0c2500 2023-10-30T12:50:47.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38864/0 (socket says 172.21.15.8:38864) 2023-10-30T12:50:47.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 -- 172.21.15.8:0/1959315022 learned_addr learned my addr 172.21.15.8:0/1959315022 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:47.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe2023a5640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 0x7fe1fc1498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe202ba6640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 0x7fe1fc145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 -- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 msgr2=0x7fe1fc1498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 0x7fe1fc1498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 -- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 msgr2=0x7fe1fc145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 0x7fe1fc145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 -- 172.21.15.8:0/1959315022 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009a70 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe2023a5640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 0x7fe1fc1498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:47.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe202ba6640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 0x7fe1fc145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:47.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe201ba4640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0011f0 tx=0x7fe1ec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0074e0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec007640 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec004a10 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc14a100 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1f1d80 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc04e170 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.342+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe1ec004b70 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.346+0000 7fe1f2ffd640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 0x7fe1e0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.346+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fe1ec0834d0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.346+0000 7fe2023a5640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 0x7fe1e0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:47.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.346+0000 7fe2023a5640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 0x7fe1e0068990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007a70 tx=0x7fe1f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:47.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.350+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe1ec0504d0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.538+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1fc1451d0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.538+0000 7fe1f2ffd640 1 -- 172.21.15.8:0/1959315022 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7fe1ec053af0 con 0x7fe1fc0c82d0 2023-10-30T12:50:47.545 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:47.546 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:47.546 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:47.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 msgr2=0x7fe1e0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 0x7fe1e0068990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007a70 tx=0x7fe1f8008040 comp rx=0 tx=0).stop 2023-10-30T12:50:47.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 msgr2=0x7fe1fc06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:47.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0011f0 tx=0x7fe1ec00c9b0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 shutdown_connections 2023-10-30T12:50:47.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1e00664d0 0x7fe1e0068990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1fc1516c0 0x7fe1fc145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1fc0c82d0 0x7fe1fc06c7e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 --2- 172.21.15.8:0/1959315022 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1fc0c2500 0x7fe1fc1498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:47.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 >> 172.21.15.8:0/1959315022 conn(0x7fe1fc071fb0 msgr2=0x7fe1fc150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:47.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 shutdown_connections 2023-10-30T12:50:47.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:47.542+0000 7fe204609640 1 -- 172.21.15.8:0/1959315022 wait complete. 2023-10-30T12:50:47.566 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:49.568 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:49.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 Processor -- start 2023-10-30T12:50:49.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 -- start start 2023-10-30T12:50:49.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c38153c80 0x7f2c3815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2c380c4a10 0x7f2c380c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c3815a3b0 0x7f2c380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3815dec0 con 0x7f2c380c4a10 2023-10-30T12:50:49.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c381599a0 con 0x7f2c3815a3b0 2023-10-30T12:50:49.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3c9d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38159f20 con 0x7f2c38153c80 2023-10-30T12:50:49.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2c380c4a10 0x7f2c380c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2c380c4a10 0x7f2c380c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37448/0 (socket says 172.21.15.8:37448) 2023-10-30T12:50:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 -- 172.21.15.8:0/987956574 learned_addr learned my addr 172.21.15.8:0/987956574 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c36d9d640 1 --2- 172.21.15.8:0/987956574 >> v2:172.21.15.46:3300/0 conn(0x7f2c3815a3b0 0x7f2c380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c3659c640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c38153c80 0x7f2c3815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 -- 172.21.15.8:0/987956574 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c38153c80 msgr2=0x7f2c3815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c38153c80 0x7f2c3815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 -- 172.21.15.8:0/987956574 >> v2:172.21.15.46:3300/0 conn(0x7f2c3815a3b0 msgr2=0x7f2c380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/987956574 >> v2:172.21.15.46:3300/0 conn(0x7f2c3815a3b0 0x7f2c380ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.718+0000 7f2c2ffff640 1 -- 172.21.15.8:0/987956574 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380c47e0 con 0x7f2c380c4a10 2023-10-30T12:50:49.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c380c42a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c380cbbc0 tx=0x7f2c2001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f13c309fc3dbe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:49.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c35d9b640 1 -- 172.21.15.8:0/987956574 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200a89b0 con 0x7f2c380c4a10 2023-10-30T12:50:49.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c35d9b640 1 -- 172.21.15.8:0/987956574 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c200a8b10 con 0x7f2c380c4a10 2023-10-30T12:50:49.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c35d9b640 1 -- 172.21.15.8:0/987956574 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200a8ea0 con 0x7f2c380c4a10 2023-10-30T12:50:49.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/987956574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 msgr2=0x7f2c380c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:49.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c380c42a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c380cbbc0 tx=0x7f2c2001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/987956574 shutdown_connections 2023-10-30T12:50:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/987956574 >> v2:172.21.15.46:3300/0 conn(0x7f2c3815a3b0 0x7f2c380ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c380c42a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/987956574 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c38153c80 0x7f2c3815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/987956574 >> 172.21.15.8:0/987956574 conn(0x7f2c380715c0 msgr2=0x7f2c380719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/987956574 shutdown_connections 2023-10-30T12:50:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.722+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/987956574 wait complete. 2023-10-30T12:50:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 Processor -- start 2023-10-30T12:50:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- start start 2023-10-30T12:50:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 0x7f2c380b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c3815a3b0 0x7f2c38149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b2b70 con 0x7f2c380c4a10 2023-10-30T12:50:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381eea50 con 0x7f2c38153c80 2023-10-30T12:50:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b2e30 con 0x7f2c3815a3b0 2023-10-30T12:50:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37462/0 (socket says 172.21.15.8:37462) 2023-10-30T12:50:49.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 -- 172.21.15.8:0/3412570567 learned_addr learned my addr 172.21.15.8:0/3412570567 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:49.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 0x7f2c380b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c36d9d640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c3815a3b0 0x7f2c38149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 -- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c3815a3b0 msgr2=0x7f2c38149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c3815a3b0 0x7f2c38149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 -- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 msgr2=0x7f2c380b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 0x7f2c380b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 -- 172.21.15.8:0/3412570567 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200094d0 con 0x7f2c380c4a10 2023-10-30T12:50:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 0x7f2c380b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3659c640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d800 tx=0x7f2c2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280096a0 con 0x7f2c380c4a10 2023-10-30T12:50:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28009800 con 0x7f2c380c4a10 2023-10-30T12:50:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c381eed10 con 0x7f2c380c4a10 2023-10-30T12:50:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28009b70 con 0x7f2c380c4a10 2023-10-30T12:50:49.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.726+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c381ef2d0 con 0x7f2c380c4a10 2023-10-30T12:50:49.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3804e170 con 0x7f2c380c4a10 2023-10-30T12:50:49.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2c28009cd0 con 0x7f2c380c4a10 2023-10-30T12:50:49.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c2effd640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 0x7f2c18068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:49.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f2c280849f0 con 0x7f2c380c4a10 2023-10-30T12:50:49.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 0x7f2c18068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:49.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.730+0000 7f2c2ffff640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 0x7f2c18068990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c200b2000 tx=0x7f2c20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:49.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.734+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2c280519f0 con 0x7f2c380c4a10 2023-10-30T12:50:49.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.922+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c381594b0 con 0x7f2c380c4a10 2023-10-30T12:50:49.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.922+0000 7f2c2effd640 1 -- 172.21.15.8:0/3412570567 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f2c28055010 con 0x7f2c380c4a10 2023-10-30T12:50:49.929 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:49.929 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:49.929 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:49.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.922+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 msgr2=0x7f2c18068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:49.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.922+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 0x7f2c18068990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c200b2000 tx=0x7f2c20008040 comp rx=0 tx=0).stop 2023-10-30T12:50:49.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.922+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 msgr2=0x7f2c38151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:49.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d800 tx=0x7f2c2800dcc0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 shutdown_connections 2023-10-30T12:50:49.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2c180664d0 0x7f2c18068990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c38153c80 0x7f2c380b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2c3815a3b0 0x7f2c38149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 --2- 172.21.15.8:0/3412570567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c380c4a10 0x7f2c38151fd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:49.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 >> 172.21.15.8:0/3412570567 conn(0x7f2c380715c0 msgr2=0x7f2c38158450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:49.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 shutdown_connections 2023-10-30T12:50:49.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:49.926+0000 7f2c3c9d3640 1 -- 172.21.15.8:0/3412570567 wait complete. 2023-10-30T12:50:49.950 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:51.951 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 Processor -- start 2023-10-30T12:50:52.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 -- start start 2023-10-30T12:50:52.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fe40c9b90 0x7f3fe406bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fe4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe415d070 con 0x7f3fe4153b10 2023-10-30T12:50:52.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe4159900 con 0x7f3fe40c9b90 2023-10-30T12:50:52.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3febe05640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40ca000 con 0x7f3fe406c480 2023-10-30T12:50:52.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe9ba1640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3fe4153b10 0x7f3fe40ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:33826/0 (socket says 172.21.15.8:33826) 2023-10-30T12:50:52.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe9ba1640 1 -- 172.21.15.8:0/84461962 learned_addr learned my addr 172.21.15.8:0/84461962 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:52.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe93a0640 1 --2- 172.21.15.8:0/84461962 >> v2:172.21.15.46:3300/0 conn(0x7f3fe40c9b90 0x7f3fe406bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821735576 0 0) 0x7f3fe415d070 con 0x7f3fe4153b10 2023-10-30T12:50:52.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fe4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd0003530 con 0x7f3fe4153b10 2023-10-30T12:50:52.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897757893 0 0) 0x7f3fd0003530 con 0x7f3fe4153b10 2023-10-30T12:50:52.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 msgr2=0x7f3fe4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fe4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 >> v2:172.21.15.46:3300/0 conn(0x7f3fe40c9b90 msgr2=0x7f3fe406bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 --2- 172.21.15.8:0/84461962 >> v2:172.21.15.46:3300/0 conn(0x7f3fe40c9b90 0x7f3fe406bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe415d350 con 0x7f3fe4153b10 2023-10-30T12:50:52.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f3fe0001680 con 0x7f3fe4153b10 2023-10-30T12:50:52.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 >> v1:172.21.15.8:6789/0 conn(0x7f3fe4153b10 legacy=0x7f3fe40ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd0003980 0x7f3fd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fd00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.098+0000 7f3fe8b9f640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fd0004d10 0x7f3fd0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe4159900 con 0x7f3fd0004d10 2023-10-30T12:50:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40ca000 con 0x7f3fd0003980 2023-10-30T12:50:52.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd000ba40 con 0x7f3fe406c480 2023-10-30T12:50:52.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fd0004d10 0x7f3fd0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe9ba1640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd0003980 0x7f3fd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe93a0640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fd00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 -- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 msgr2=0x7f3fd00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fe406c480 0x7f3fd00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 -- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd0003980 msgr2=0x7f3fd0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd0003980 0x7f3fd0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 -- 172.21.15.8:0/84461962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0003530 con 0x7f3fd0004d10 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe9ba1640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd0003980 0x7f3fd0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fd0004d10 0x7f3fd0005190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007360 tx=0x7f3fdc01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8f6ca0500cde0d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0a5d20 con 0x7f3fd0004d10 2023-10-30T12:50:52.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0a5e80 con 0x7f3fd0004d10 2023-10-30T12:50:52.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.102+0000 7f3fe8b9f640 1 -- 172.21.15.8:0/84461962 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0ad660 con 0x7f3fd0004d10 2023-10-30T12:50:52.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- 172.21.15.8:0/84461962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fd0004d10 msgr2=0x7f3fd0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 --2- 172.21.15.8:0/84461962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fd0004d10 0x7f3fd0005190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007360 tx=0x7f3fdc01ce20 comp rx=0 tx=0).stop 2023-10-30T12:50:52.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 -- 172.21.15.8:0/84461962 reap_dead start 2023-10-30T12:50:52.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- 172.21.15.8:0/84461962 shutdown_connections 2023-10-30T12:50:52.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- 172.21.15.8:0/84461962 >> 172.21.15.8:0/84461962 conn(0x7f3fe406a240 msgr2=0x7f3fe406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:52.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- 172.21.15.8:0/84461962 shutdown_connections 2023-10-30T12:50:52.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- 172.21.15.8:0/84461962 wait complete. 2023-10-30T12:50:52.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 Processor -- start 2023-10-30T12:50:52.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- start start 2023-10-30T12:50:52.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fd0004d10 0x7f3fe40c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe40b3940 0x7f3fe415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe415bfc0 con 0x7f3fe415c920 2023-10-30T12:50:52.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41f2190 con 0x7f3fe40b3940 2023-10-30T12:50:52.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3febe05640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41f3120 con 0x7f3fd0004d10 2023-10-30T12:50:52.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37484/0 (socket says 172.21.15.8:37484) 2023-10-30T12:50:52.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 -- 172.21.15.8:0/3219129555 learned_addr learned my addr 172.21.15.8:0/3219129555 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:52.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fe9ba1640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fd0004d10 0x7f3fe40c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fe93a0640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe40b3940 0x7f3fe415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 -- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fd0004d10 msgr2=0x7f3fe40c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fd0004d10 0x7f3fe40c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 -- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe40b3940 msgr2=0x7f3fe415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe40b3940 0x7f3fe415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.106+0000 7f3fea3a2640 1 -- 172.21.15.8:0/3219129555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4152460 con 0x7f3fe415c920 2023-10-30T12:50:52.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fea3a2640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc002240 tx=0x7f3fd0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0adb30 con 0x7f3fe415c920 2023-10-30T12:50:52.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0adcb0 con 0x7f3fe415c920 2023-10-30T12:50:52.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0b86b0 con 0x7f3fe415c920 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe41521a0 con 0x7f3fe415c920 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe40cfed0 con 0x7f3fe415c920 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe40cfd30 con 0x7f3fe415c920 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3fdc0b4070 con 0x7f3fe415c920 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 0x7f3fc8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:52.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f3fdc11e330 con 0x7f3fe415c920 2023-10-30T12:50:52.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fe9ba1640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 0x7f3fc8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:52.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.110+0000 7f3fe9ba1640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 0x7f3fc8068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c10 tx=0x7f3fe0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:52.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.114+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3fe40cfd30 con 0x7f3fe415c920 2023-10-30T12:50:52.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe415bd20 con 0x7f3fe415c920 2023-10-30T12:50:52.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3fda7fc640 1 -- 172.21.15.8:0/3219129555 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f3fe415bd20 con 0x7f3fe415c920 2023-10-30T12:50:52.312 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:52.312 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:52.313 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:52.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 msgr2=0x7f3fc8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 0x7f3fc8068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c10 tx=0x7f3fe0009040 comp rx=0 tx=0).stop 2023-10-30T12:50:52.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 msgr2=0x7f3fe40c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:52.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc002240 tx=0x7f3fd0005800 comp rx=0 tx=0).stop 2023-10-30T12:50:52.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 shutdown_connections 2023-10-30T12:50:52.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3fc80664a0 0x7f3fc8068960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe40b3940 0x7f3fe415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe415c920 0x7f3fe40c7b80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 --2- 172.21.15.8:0/3219129555 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3fd0004d10 0x7f3fe40c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:52.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 >> 172.21.15.8:0/3219129555 conn(0x7f3fe406a240 msgr2=0x7f3fe4152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:52.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 shutdown_connections 2023-10-30T12:50:52.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:52.306+0000 7f3febe05640 1 -- 172.21.15.8:0/3219129555 wait complete. 2023-10-30T12:50:52.333 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:54.334 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:54.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.482+0000 7fcfc0c46640 1 Processor -- start 2023-10-30T12:50:54.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.482+0000 7fcfc0c46640 1 -- start start 2023-10-30T12:50:54.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.482+0000 7fcfc0c46640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcfbc153aa0 0x7fcfbc15d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.482+0000 7fcfc0c46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfc0c46640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc0c7b00 con 0x7fcfbc0701e0 2023-10-30T12:50:54.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfc0c46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc070a50 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfc0c46640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc15dd80 con 0x7fcfbc153aa0 2023-10-30T12:50:54.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37490/0 (socket says 172.21.15.8:37490) 2023-10-30T12:50:54.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 -- 172.21.15.8:0/1586316755 learned_addr learned my addr 172.21.15.8:0/1586316755 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:54.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfb3fff640 1 --1- 172.21.15.8:0/1586316755 >> v1:172.21.15.8:6789/0 conn(0x7fcfbc0701e0 0x7fcfbc0ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:33828/0 (socket says 172.21.15.8:33828) 2023-10-30T12:50:54.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfba59c640 1 --2- 172.21.15.8:0/1586316755 >> v2:172.21.15.46:3300/0 conn(0x7fcfbc153aa0 0x7fcfbc15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 -- 172.21.15.8:0/1586316755 >> v2:172.21.15.46:3300/0 conn(0x7fcfbc153aa0 msgr2=0x7fcfbc15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:54.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 --2- 172.21.15.8:0/1586316755 >> v2:172.21.15.46:3300/0 conn(0x7fcfbc153aa0 0x7fcfbc15d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 -- 172.21.15.8:0/1586316755 >> v1:172.21.15.8:6789/0 conn(0x7fcfbc0701e0 legacy=0x7fcfbc0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 -- 172.21.15.8:0/1586316755 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc06bd20 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfbad9d640 1 --2- 172.21.15.8:0/1586316755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0caae0 tx=0x7fcfb401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed4fa9305cd6e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:54.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfb9d9b640 1 -- 172.21.15.8:0/1586316755 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40af400 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfb9d9b640 1 -- 172.21.15.8:0/1586316755 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb40ab050 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.486+0000 7fcfb9d9b640 1 -- 172.21.15.8:0/1586316755 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40af780 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- 172.21.15.8:0/1586316755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 msgr2=0x7fcfbc06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/1586316755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0caae0 tx=0x7fcfb401f1c0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- 172.21.15.8:0/1586316755 shutdown_connections 2023-10-30T12:50:54.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/1586316755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc06b7e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/1586316755 >> v2:172.21.15.46:3300/0 conn(0x7fcfbc153aa0 0x7fcfbc15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- 172.21.15.8:0/1586316755 >> 172.21.15.8:0/1586316755 conn(0x7fcfbc071fb0 msgr2=0x7fcfbc0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:54.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- 172.21.15.8:0/1586316755 shutdown_connections 2023-10-30T12:50:54.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- 172.21.15.8:0/1586316755 wait complete. 2023-10-30T12:50:54.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 Processor -- start 2023-10-30T12:50:54.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- start start 2023-10-30T12:50:54.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc0701e0 0x7fcfbc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcfbc153aa0 0x7fcfbc159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc154610 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1548f0 con 0x7fcfbc0701e0 2023-10-30T12:50:54.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfc0c46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0ca350 con 0x7fcfbc153aa0 2023-10-30T12:50:54.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfba59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfba59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37506/0 (socket says 172.21.15.8:37506) 2023-10-30T12:50:54.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfba59c640 1 -- 172.21.15.8:0/969802 learned_addr learned my addr 172.21.15.8:0/969802 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:54.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfb3fff640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc0701e0 0x7fcfbc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.490+0000 7fcfbad9d640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcfbc153aa0 0x7fcfbc159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 -- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcfbc153aa0 msgr2=0x7fcfbc159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcfbc153aa0 0x7fcfbc159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 -- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc0701e0 msgr2=0x7fcfbc156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc0701e0 0x7fcfbc156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 -- 172.21.15.8:0/969802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4009b30 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfba59c640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d5f0 tx=0x7fcfac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:54.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0125f0 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac012750 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc0ca630 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac012ae0 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc0c9af0 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc155ba0 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcfac012c40 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 0x7fcf9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:54.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fcfac084770 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.494+0000 7fcfb3fff640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 0x7fcf9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:54.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.498+0000 7fcfb3fff640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 0x7fcf9c068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc06bcf0 tx=0x7fcfa4007cb0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:54.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.498+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcfac051800 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.690+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfbc0ca820 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfb2ffd640 1 -- 172.21.15.8:0/969802 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7fcfac007380 con 0x7fcfbc06b3b0 2023-10-30T12:50:54.699 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:54.699 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:54.700 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:54.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 msgr2=0x7fcf9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 0x7fcf9c068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc06bcf0 tx=0x7fcfa4007cb0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 msgr2=0x7fcfbc155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:54.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d5f0 tx=0x7fcfac00dab0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 shutdown_connections 2023-10-30T12:50:54.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcf9c0664a0 0x7fcf9c068960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc0701e0 0x7fcfbc156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcfbc153aa0 0x7fcfbc159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 --2- 172.21.15.8:0/969802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcfbc06b3b0 0x7fcfbc155d50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:54.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 >> 172.21.15.8:0/969802 conn(0x7fcfbc071fb0 msgr2=0x7fcfbc0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:54.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 shutdown_connections 2023-10-30T12:50:54.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:54.694+0000 7fcfc0c46640 1 -- 172.21.15.8:0/969802 wait complete. 2023-10-30T12:50:54.720 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:56.720 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:56.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 Processor -- start 2023-10-30T12:50:56.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 -- start start 2023-10-30T12:50:56.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e38153c80 0x7f3e3815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e38159f20 con 0x7f3e380c4a10 2023-10-30T12:50:56.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3815dec0 con 0x7f3e38153c80 2023-10-30T12:50:56.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e3d30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381599a0 con 0x7f3e380c42a0 2023-10-30T12:50:56.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e367fc640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3e380c4a10 0x7f3e3815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:33836/0 (socket says 172.21.15.8:33836) 2023-10-30T12:50:56.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e367fc640 1 -- 172.21.15.8:0/2376956744 learned_addr learned my addr 172.21.15.8:0/2376956744 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:56.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e377fe640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/2376956744 >> v2:172.21.15.46:3300/0 conn(0x7f3e38153c80 0x7f3e3815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987191632 0 0) 0x7f3e38159f20 con 0x7f3e380c4a10 2023-10-30T12:50:56.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e20003530 con 0x7f3e380c4a10 2023-10-30T12:50:56.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032002960 0 0) 0x7f3e20003530 con 0x7f3e380c4a10 2023-10-30T12:50:56.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 msgr2=0x7f3e380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:56.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e380ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 >> v2:172.21.15.46:3300/0 conn(0x7f3e38153c80 msgr2=0x7f3e3815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:56.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 --2- 172.21.15.8:0/2376956744 >> v2:172.21.15.46:3300/0 conn(0x7f3e38153c80 0x7f3e3815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3815d620 con 0x7f3e380c4a10 2023-10-30T12:50:56.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f3e2c001680 con 0x7f3e380c4a10 2023-10-30T12:50:56.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/2376956744 >> v2:172.21.15.46:3300/0 conn(0x7f3e38153c80 0x7f3e3815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.870+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 >> v1:172.21.15.8:6789/0 conn(0x7f3e380c4a10 legacy=0x7f3e3815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e20003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e20003f00 0x7f3e200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e20004d10 0x7f3e20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3815dec0 con 0x7f3e20003f00 2023-10-30T12:50:56.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381599a0 con 0x7f3e20004d10 2023-10-30T12:50:56.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2000b9c0 con 0x7f3e380c42a0 2023-10-30T12:50:56.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e20003f00 0x7f3e200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e20003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e377fe640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e20004d10 0x7f3e20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 -- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 msgr2=0x7f3e20003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:56.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e380c42a0 0x7f3e20003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 -- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e20004d10 msgr2=0x7f3e20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:56.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e20004d10 0x7f3e20005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 -- 172.21.15.8:0/2376956744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20003530 con 0x7f3e20003f00 2023-10-30T12:50:56.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e367fc640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e20003f00 0x7f3e200047d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c210 tx=0x7f3e2c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9543cc04d74b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:56.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0a5940 con 0x7f3e20003f00 2023-10-30T12:50:56.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0a5aa0 con 0x7f3e20003f00 2023-10-30T12:50:56.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.874+0000 7f3e35ffb640 1 -- 172.21.15.8:0/2376956744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0058e0 con 0x7f3e20003f00 2023-10-30T12:50:56.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- 172.21.15.8:0/2376956744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e20003f00 msgr2=0x7f3e200047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:56.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/2376956744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e20003f00 0x7f3e200047d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c210 tx=0x7f3e2c01cc40 comp rx=0 tx=0).stop 2023-10-30T12:50:56.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e377fe640 1 -- 172.21.15.8:0/2376956744 reap_dead start 2023-10-30T12:50:56.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- 172.21.15.8:0/2376956744 shutdown_connections 2023-10-30T12:50:56.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- 172.21.15.8:0/2376956744 >> 172.21.15.8:0/2376956744 conn(0x7f3e380715c0 msgr2=0x7f3e380719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:56.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- 172.21.15.8:0/2376956744 shutdown_connections 2023-10-30T12:50:56.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- 172.21.15.8:0/2376956744 wait complete. 2023-10-30T12:50:56.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 Processor -- start 2023-10-30T12:50:56.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- start start 2023-10-30T12:50:56.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e20003f00 0x7f3e380ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 0x7f3e381499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380ce7a0 con 0x7f3e3814a860 2023-10-30T12:50:56.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380cea80 con 0x7f3e3815cf80 2023-10-30T12:50:56.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e3d30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3814c420 con 0x7f3e20003f00 2023-10-30T12:50:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37528/0 (socket says 172.21.15.8:37528) 2023-10-30T12:50:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e367fc640 1 -- 172.21.15.8:0/3898332655 learned_addr learned my addr 172.21.15.8:0/3898332655 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e20003f00 0x7f3e380ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.878+0000 7f3e377fe640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 0x7f3e381499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 -- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e20003f00 msgr2=0x7f3e380ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e20003f00 0x7f3e380ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 -- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 msgr2=0x7f3e381499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 0x7f3e381499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:56.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 -- 172.21.15.8:0/3898332655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3814c700 con 0x7f3e3814a860 2023-10-30T12:50:56.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e377fe640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 0x7f3e381499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:50:56.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e367fc640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0ab040 tx=0x7f3e20005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:56.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c001c60 con 0x7f3e3814a860 2023-10-30T12:50:56.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0b3070 con 0x7f3e3814a860 2023-10-30T12:50:56.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0072f0 con 0x7f3e3814a860 2023-10-30T12:50:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3814c9e0 con 0x7f3e3814a860 2023-10-30T12:50:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380ce200 con 0x7f3e3814a860 2023-10-30T12:50:56.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3814cb70 con 0x7f3e3814a860 2023-10-30T12:50:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3e2c007450 con 0x7f3e3814a860 2023-10-30T12:50:56.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.882+0000 7f3e1f7fe640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 0x7f3e24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.886+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f3e2c0b5070 con 0x7f3e3814a860 2023-10-30T12:50:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.886+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 0x7f3e24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.886+0000 7f3e36ffd640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 0x7f3e24068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e280093f0 tx=0x7f3e28009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:56.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:56.890+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3e2c0eb6c0 con 0x7f3e3814a860 2023-10-30T12:50:57.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.074+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e38159b80 con 0x7f3e3814a860 2023-10-30T12:50:57.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e1f7fe640 1 -- 172.21.15.8:0/3898332655 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f3e2c0eece0 con 0x7f3e3814a860 2023-10-30T12:50:57.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:57.084 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5079,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:10.188288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":4276,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"f","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6833/3888673612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3888673612},{"type":"v1","addr":"172.21.15.46:6833","nonce":3888673612}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:57.084 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2023-10-30T12:50:57.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 msgr2=0x7f3e24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 0x7f3e24068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e280093f0 tx=0x7f3e28009190 comp rx=0 tx=0).stop 2023-10-30T12:50:57.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 msgr2=0x7f3e380cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:57.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0ab040 tx=0x7f3e20005800 comp rx=0 tx=0).stop 2023-10-30T12:50:57.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 shutdown_connections 2023-10-30T12:50:57.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3e240664a0 0x7f3e24068960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3815cf80 0x7f3e381499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e3814a860 0x7f3e380cede0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 --2- 172.21.15.8:0/3898332655 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3e20003f00 0x7f3e380ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:57.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 >> 172.21.15.8:0/3898332655 conn(0x7f3e380715c0 msgr2=0x7f3e380cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:57.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 shutdown_connections 2023-10-30T12:50:57.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:57.078+0000 7f3e3d30f640 1 -- 172.21.15.8:0/3898332655 wait complete. 2023-10-30T12:50:57.104 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-30T12:50:59.106 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:59.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.254+0000 7fa79c468640 1 Processor -- start 2023-10-30T12:50:59.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.254+0000 7fa79c468640 1 -- start start 2023-10-30T12:50:59.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa794150830 0x7fa79406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7940cb7d0 0x7fa7940c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 0x7fa794070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940c2310 con 0x7fa794150830 2023-10-30T12:50:59.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa794156710 con 0x7fa7940cb7d0 2023-10-30T12:50:59.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79c468640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940c4360 con 0x7fa7940c3e40 2023-10-30T12:50:59.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79aa05640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 0x7fa794070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79aa05640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 0x7fa794070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:50762/0 (socket says 172.21.15.8:50762) 2023-10-30T12:50:59.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79aa05640 1 -- 172.21.15.8:0/2074170564 learned_addr learned my addr 172.21.15.8:0/2074170564 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:59.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 --2- 172.21.15.8:0/2074170564 >> v2:172.21.15.8:3300/0 conn(0x7fa794150830 0x7fa79406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa799a03640 1 --2- 172.21.15.8:0/2074170564 >> v2:172.21.15.46:3300/0 conn(0x7fa7940cb7d0 0x7fa7940c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 -- 172.21.15.8:0/2074170564 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 msgr2=0x7fa794070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:59.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 --2- 172.21.15.8:0/2074170564 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 0x7fa794070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 -- 172.21.15.8:0/2074170564 >> v2:172.21.15.46:3300/0 conn(0x7fa7940cb7d0 msgr2=0x7fa7940c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:59.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 --2- 172.21.15.8:0/2074170564 >> v2:172.21.15.46:3300/0 conn(0x7fa7940cb7d0 0x7fa7940c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.258+0000 7fa79a204640 1 -- 172.21.15.8:0/2074170564 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7941569b0 con 0x7fa794150830 2023-10-30T12:50:59.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79a204640 1 --2- 172.21.15.8:0/2074170564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa79406b500 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa79406f9f0 tx=0x7fa79001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcc31d7e68b51f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa799202640 1 -- 172.21.15.8:0/2074170564 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900a8910 con 0x7fa794150830 2023-10-30T12:50:59.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa799202640 1 -- 172.21.15.8:0/2074170564 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7900a8a70 con 0x7fa794150830 2023-10-30T12:50:59.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa799202640 1 -- 172.21.15.8:0/2074170564 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900a8e00 con 0x7fa794150830 2023-10-30T12:50:59.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 -- 172.21.15.8:0/2074170564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 msgr2=0x7fa79406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 --2- 172.21.15.8:0/2074170564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa79406b500 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa79406f9f0 tx=0x7fa79001ec50 comp rx=0 tx=0).stop 2023-10-30T12:50:59.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 -- 172.21.15.8:0/2074170564 shutdown_connections 2023-10-30T12:50:59.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 --2- 172.21.15.8:0/2074170564 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940c3e40 0x7fa794070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 --2- 172.21.15.8:0/2074170564 >> v2:172.21.15.46:3300/0 conn(0x7fa7940cb7d0 0x7fa7940c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 --2- 172.21.15.8:0/2074170564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa79406b500 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 -- 172.21.15.8:0/2074170564 >> 172.21.15.8:0/2074170564 conn(0x7fa7940cac70 msgr2=0x7fa7940cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:59.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 -- 172.21.15.8:0/2074170564 shutdown_connections 2023-10-30T12:50:59.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 -- 172.21.15.8:0/2074170564 wait complete. 2023-10-30T12:50:59.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.262+0000 7fa79c468640 1 Processor -- start 2023-10-30T12:50:59.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- start start 2023-10-30T12:50:59.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7940c3e40 0x7fa794157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940cb7d0 0x7fa794152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794152b80 con 0x7fa794150830 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79415ab10 con 0x7fa7940c3e40 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79414ff10 con 0x7fa7940cb7d0 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37512/0 (socket says 172.21.15.8:37512) 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 -- 172.21.15.8:0/2167748459 learned_addr learned my addr 172.21.15.8:0/2167748459 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:59.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa799a03640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940cb7d0 0x7fa794152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79a204640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7940c3e40 0x7fa794157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 -- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940cb7d0 msgr2=0x7fa794152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940cb7d0 0x7fa794152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 -- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7940c3e40 msgr2=0x7fa794157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7940c3e40 0x7fa794157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 -- 172.21.15.8:0/2167748459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009420 con 0x7fa794150830 2023-10-30T12:50:59.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79aa05640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00eed0 tx=0x7fa78c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c010040 con 0x7fa794150830 2023-10-30T12:50:59.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c00bb10 con 0x7fa794150830 2023-10-30T12:50:59.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79415a550 con 0x7fa794150830 2023-10-30T12:50:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0039a0 con 0x7fa794150830 2023-10-30T12:50:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa794157e10 con 0x7fa794150830 2023-10-30T12:50:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.266+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa794152cc0 con 0x7fa794150830 2023-10-30T12:50:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.270+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa78c003b00 con 0x7fa794150830 2023-10-30T12:50:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.270+0000 7fa78affd640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 0x7fa778068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.270+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fa78c083f60 con 0x7fa794150830 2023-10-30T12:50:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.270+0000 7fa79a204640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 0x7fa778068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.270+0000 7fa79a204640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 0x7fa778068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7900b2000 tx=0x7fa790008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.274+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa794152cc0 con 0x7fa794150830 2023-10-30T12:50:59.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa794152cc0 con 0x7fa794150830 2023-10-30T12:50:59.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa78affd640 1 -- 172.21.15.8:0/2167748459 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5729 (secure 0 0 0) 0x7fa794152cc0 con 0x7fa794150830 2023-10-30T12:50:59.472 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:59.473 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:57.122006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:59.473 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 31 2023-10-30T12:50:59.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 msgr2=0x7fa778068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 0x7fa778068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7900b2000 tx=0x7fa790008040 comp rx=0 tx=0).stop 2023-10-30T12:50:59.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 msgr2=0x7fa794151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00eed0 tx=0x7fa78c00b480 comp rx=0 tx=0).stop 2023-10-30T12:50:59.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 shutdown_connections 2023-10-30T12:50:59.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa7780664a0 0x7fa778068960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa794150830 0x7fa794151730 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa7940cb7d0 0x7fa794152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 --2- 172.21.15.8:0/2167748459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7940c3e40 0x7fa794157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 >> 172.21.15.8:0/2167748459 conn(0x7fa7940cac70 msgr2=0x7fa79406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.466+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 shutdown_connections 2023-10-30T12:50:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.474+0000 7fa79c468640 1 -- 172.21.15.8:0/2167748459 wait complete. 2023-10-30T12:50:59.494 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-10-30T12:50:59.494 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:50:59.494 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:50:59.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.634+0000 7f1588f8d640 1 Processor -- start 2023-10-30T12:50:59.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.634+0000 7f1588f8d640 1 -- start start 2023-10-30T12:50:59.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1588f8d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f15840c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1588f8d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840c5030 con 0x7f1584151840 2023-10-30T12:50:59.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1588f8d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1584157830 con 0x7f1584151c20 2023-10-30T12:50:59.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1588f8d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1584157b10 con 0x7f158406bd10 2023-10-30T12:50:59.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158259c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f1584151840 0x7f1584072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:49966/0 (socket says 172.21.15.8:49966) 2023-10-30T12:50:59.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158259c640 1 -- 172.21.15.8:0/4148411913 learned_addr learned my addr 172.21.15.8:0/4148411913 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:50:59.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f15840c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167538862 0 0) 0x7f15840c5030 con 0x7f1584151840 2023-10-30T12:50:59.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c003530 con 0x7f1584151840 2023-10-30T12:50:59.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781062667 0 0) 0x7f156c003530 con 0x7f1584151840 2023-10-30T12:50:59.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 msgr2=0x7f15840c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:59.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f15840c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 >> v1:172.21.15.46:6789/0 conn(0x7f1584151c20 legacy=0x7f158406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584157df0 con 0x7f1584151840 2023-10-30T12:50:59.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f1574001680 con 0x7f1584151840 2023-10-30T12:50:59.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 >> v1:172.21.15.8:6789/0 conn(0x7f1584151840 legacy=0x7f1584072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f156c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f156c003f30 0x7f156c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f156c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584157b10 con 0x7f156c004d10 2023-10-30T12:50:59.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00bb00 con 0x7f156c003f30 2023-10-30T12:50:59.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00bcc0 con 0x7f158406bd10 2023-10-30T12:50:59.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f156c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1581d9b640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f156c003f30 0x7f156c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f158259c640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f156c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 -- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 msgr2=0x7f156c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:59.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f158406bd10 0x7f156c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 -- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f156c003f30 msgr2=0x7f156c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:50:59.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f156c003f30 0x7f156c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.638+0000 7f1582d9d640 1 -- 172.21.15.8:0/4148411913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c003530 con 0x7f156c004d10 2023-10-30T12:50:59.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1582d9d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f156c005190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f157c009fc0 tx=0x7f157c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe702fcaf4e688ce server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0a5c70 con 0x7f156c004d10 2023-10-30T12:50:59.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00f040 con 0x7f156c004d10 2023-10-30T12:50:59.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f158159a640 1 -- 172.21.15.8:0/4148411913 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0ad440 con 0x7f156c004d10 2023-10-30T12:50:59.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 -- 172.21.15.8:0/4148411913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 msgr2=0x7f156c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 --2- 172.21.15.8:0/4148411913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f156c005190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f157c009fc0 tx=0x7f157c01cc60 comp rx=0 tx=0).stop 2023-10-30T12:50:59.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1582d9d640 1 -- 172.21.15.8:0/4148411913 reap_dead start 2023-10-30T12:50:59.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 -- 172.21.15.8:0/4148411913 shutdown_connections 2023-10-30T12:50:59.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 -- 172.21.15.8:0/4148411913 >> 172.21.15.8:0/4148411913 conn(0x7f15840cac70 msgr2=0x7f15840cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:59.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 -- 172.21.15.8:0/4148411913 shutdown_connections 2023-10-30T12:50:59.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 -- 172.21.15.8:0/4148411913 wait complete. 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.642+0000 7f1588f8d640 1 Processor -- start 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- start start 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15840ca320 0x7f15841ec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f15840c3450 0x7f15841ece20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841e1a10 con 0x7f156c004d10 2023-10-30T12:50:59.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841ec490 con 0x7f15840ca320 2023-10-30T12:50:59.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840ca880 con 0x7f15840c3450 2023-10-30T12:50:59.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37542/0 (socket says 172.21.15.8:37542) 2023-10-30T12:50:59.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 -- 172.21.15.8:0/2260903617 learned_addr learned my addr 172.21.15.8:0/2260903617 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:50:59.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1581d9b640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15840ca320 0x7f15841ec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1582d9d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f15840c3450 0x7f15841ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 -- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f15840c3450 msgr2=0x7f15841ece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f15840c3450 0x7f15841ece20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 -- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15840ca320 msgr2=0x7f15841ec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15840ca320 0x7f15841ec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 -- 172.21.15.8:0/2260903617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15841e0e90 con 0x7f156c004d10 2023-10-30T12:50:59.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f158259c640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f157400e8c0 tx=0x7f157400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157400c870 con 0x7f156c004d10 2023-10-30T12:50:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157400c9d0 con 0x7f156c004d10 2023-10-30T12:50:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157400cd80 con 0x7f156c004d10 2023-10-30T12:50:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15841e1170 con 0x7f156c004d10 2023-10-30T12:50:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841e1730 con 0x7f156c004d10 2023-10-30T12:50:59.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.646+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158404e170 con 0x7f156c004d10 2023-10-30T12:50:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.650+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f157401d460 con 0x7f156c004d10 2023-10-30T12:50:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.650+0000 7f1572ffd640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 0x7f1564068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:50:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.650+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f15740841f0 con 0x7f156c004d10 2023-10-30T12:50:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.650+0000 7f1581d9b640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 0x7f1564068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:50:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.650+0000 7f1581d9b640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 0x7f1564068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15841ea0c0 tx=0x7f1578009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:50:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.654+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f15740513d0 con 0x7f156c004d10 2023-10-30T12:50:59.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.846+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15841580a0 con 0x7f156c004d10 2023-10-30T12:50:59.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.846+0000 7f1572ffd640 1 -- 172.21.15.8:0/2260903617 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5729 (secure 0 0 0) 0x7f15740549f0 con 0x7f156c004d10 2023-10-30T12:50:59.849 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:50:59.849 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:57.122006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:50:59.850 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 31 2023-10-30T12:50:59.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 msgr2=0x7f1564068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 0x7f1564068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15841ea0c0 tx=0x7f1578009040 comp rx=0 tx=0).stop 2023-10-30T12:50:59.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 msgr2=0x7f158414e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:50:59.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f157400e8c0 tx=0x7f157400ed80 comp rx=0 tx=0).stop 2023-10-30T12:50:59.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 shutdown_connections 2023-10-30T12:50:59.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f15640664a0 0x7f1564068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15840ca320 0x7f15841ec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f15840c3450 0x7f15841ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 --2- 172.21.15.8:0/2260903617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f156c004d10 0x7f158414e160 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:50:59.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 >> 172.21.15.8:0/2260903617 conn(0x7f15840cac70 msgr2=0x7f15840cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:50:59.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 shutdown_connections 2023-10-30T12:50:59.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:50:59.850+0000 7f1588f8d640 1 -- 172.21.15.8:0/2260903617 wait complete. 2023-10-30T12:50:59.870 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 31, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 31, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 33 }, "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:50:57.122006+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 5079, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:51:01.872 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:02.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 Processor -- start 2023-10-30T12:51:02.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 -- start start 2023-10-30T12:51:02.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 0x7f134406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f13440706a0 0x7f13440c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134406fc30 con 0x7f13440c5460 2023-10-30T12:51:02.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134415b170 con 0x7f13440706a0 2023-10-30T12:51:02.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f134a674640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134415b360 con 0x7f134406dfc0 2023-10-30T12:51:02.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f13440706a0 0x7f13440c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f13440706a0 0x7f13440c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37554/0 (socket says 172.21.15.8:37554) 2023-10-30T12:51:02.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 -- 172.21.15.8:0/1966372367 learned_addr learned my addr 172.21.15.8:0/1966372367 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:02.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1348e71640 1 --1- 172.21.15.8:0/1966372367 >> v1:172.21.15.8:6789/0 conn(0x7f13440c5460 0x7f134415aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:49976/0 (socket says 172.21.15.8:49976) 2023-10-30T12:51:02.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349672640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 0x7f134406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 -- 172.21.15.8:0/1966372367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 msgr2=0x7f134406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:02.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 0x7f134406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 -- 172.21.15.8:0/1966372367 >> v1:172.21.15.8:6789/0 conn(0x7f13440c5460 legacy=0x7f134415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349e73640 1 -- 172.21.15.8:0/1966372367 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134415b620 con 0x7f13440706a0 2023-10-30T12:51:02.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.026+0000 7f1349672640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 0x7f134406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:02.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f1349e73640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13440706a0 0x7f13440c40f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f133c009e50 tx=0x7f133c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d87da602e402e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f133bfff640 1 -- 172.21.15.8:0/1966372367 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c0af400 con 0x7f13440706a0 2023-10-30T12:51:02.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f133bfff640 1 -- 172.21.15.8:0/1966372367 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c0ab050 con 0x7f13440706a0 2023-10-30T12:51:02.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f133bfff640 1 -- 172.21.15.8:0/1966372367 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c0af780 con 0x7f13440706a0 2023-10-30T12:51:02.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 -- 172.21.15.8:0/1966372367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13440706a0 msgr2=0x7f13440c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13440706a0 0x7f13440c40f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f133c009e50 tx=0x7f133c01f1f0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 -- 172.21.15.8:0/1966372367 shutdown_connections 2023-10-30T12:51:02.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13440706a0 0x7f13440c40f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 --2- 172.21.15.8:0/1966372367 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f134406dfc0 0x7f134406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.034 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 -- 172.21.15.8:0/1966372367 >> 172.21.15.8:0/1966372367 conn(0x7f13440caae0 msgr2=0x7f13440caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:02.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 -- 172.21.15.8:0/1966372367 shutdown_connections 2023-10-30T12:51:02.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 -- 172.21.15.8:0/1966372367 wait complete. 2023-10-30T12:51:02.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.030+0000 7f134a674640 1 Processor -- start 2023-10-30T12:51:02.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- start start 2023-10-30T12:51:02.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 0x7f13441564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f13440c5460 0x7f134414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1344158530 con 0x7f134406dfc0 2023-10-30T12:51:02.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134415adb0 con 0x7f13440706a0 2023-10-30T12:51:02.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134414ccf0 con 0x7f13440c5460 2023-10-30T12:51:02.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37556/0 (socket says 172.21.15.8:37556) 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 -- 172.21.15.8:0/1263504844 learned_addr learned my addr 172.21.15.8:0/1263504844 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349e73640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f13440c5460 0x7f134414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1348e71640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 0x7f13441564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 -- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f13440c5460 msgr2=0x7f134414b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f13440c5460 0x7f134414b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 -- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 msgr2=0x7f13441564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 0x7f13441564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 -- 172.21.15.8:0/1263504844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c009b30 con 0x7f134406dfc0 2023-10-30T12:51:02.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1348e71640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 0x7f13441564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:51:02.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1349672640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f134000d800 tx=0x7f134000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400096c0 con 0x7f134406dfc0 2023-10-30T12:51:02.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340009820 con 0x7f134406dfc0 2023-10-30T12:51:02.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340009bd0 con 0x7f134406dfc0 2023-10-30T12:51:02.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13441ecc40 con 0x7f134406dfc0 2023-10-30T12:51:02.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1344158790 con 0x7f134406dfc0 2023-10-30T12:51:02.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.034+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134404e170 con 0x7f134406dfc0 2023-10-30T12:51:02.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.038+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1340009d30 con 0x7f134406dfc0 2023-10-30T12:51:02.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.038+0000 7f1339ffb640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 0x7f1328068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:02.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.038+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f1340084820 con 0x7f134406dfc0 2023-10-30T12:51:02.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.038+0000 7f1348e71640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 0x7f1328068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:02.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.038+0000 7f1348e71640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 0x7f1328068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1334007fd0 tx=0x7f1334009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:02.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.042+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1340051a00 con 0x7f134406dfc0 2023-10-30T12:51:02.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.242+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f134415b8d0 con 0x7f134406dfc0 2023-10-30T12:51:02.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.242+0000 7f1339ffb640 1 -- 172.21.15.8:0/1263504844 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5729 (secure 0 0 0) 0x7f1340055020 con 0x7f134406dfc0 2023-10-30T12:51:02.246 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:02.246 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:50:57.122006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:02.247 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 31 2023-10-30T12:51:02.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 msgr2=0x7f1328068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 0x7f1328068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1334007fd0 tx=0x7f1334009040 comp rx=0 tx=0).stop 2023-10-30T12:51:02.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 msgr2=0x7f1344155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:02.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f134000d800 tx=0x7f134000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 shutdown_connections 2023-10-30T12:51:02.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13280664a0 0x7f1328068960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13440706a0 0x7f13441564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f13440c5460 0x7f134414b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.249 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 --2- 172.21.15.8:0/1263504844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f134406dfc0 0x7f1344155b00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:02.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 >> 172.21.15.8:0/1263504844 conn(0x7f13440caae0 msgr2=0x7f13440c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:02.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 shutdown_connections 2023-10-30T12:51:02.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:02.246+0000 7f134a674640 1 -- 172.21.15.8:0/1263504844 wait complete. 2023-10-30T12:51:02.268 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:04.270 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:04.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 Processor -- start 2023-10-30T12:51:04.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 -- start start 2023-10-30T12:51:04.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0a18153c80 0x7f0a1815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a180c4a10 0x7f0a180c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a18159f20 con 0x7f0a1815a3b0 2023-10-30T12:51:04.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1815dec0 con 0x7f0a18153c80 2023-10-30T12:51:04.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a1e0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181599a0 con 0x7f0a180c4a10 2023-10-30T12:51:04.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a177fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0a18153c80 0x7f0a1815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a17fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0a1815a3b0 0x7f0a180ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:49992/0 (socket says 172.21.15.8:49992) 2023-10-30T12:51:04.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a17fff640 1 -- 172.21.15.8:0/2736366999 learned_addr learned my addr 172.21.15.8:0/2736366999 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:04.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a0ffff640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a180c4a10 0x7f0a180c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a177fe640 1 -- 172.21.15.8:0/2736366999 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a180c4a10 msgr2=0x7f0a180c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:04.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a177fe640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a180c4a10 0x7f0a180c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a177fe640 1 -- 172.21.15.8:0/2736366999 >> v1:172.21.15.8:6789/0 conn(0x7f0a1815a3b0 legacy=0x7f0a180ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a16ffd640 1 -- 172.21.15.8:0/2736366999 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162680404 0 0) 0x7f0a18159f20 con 0x7f0a1815a3b0 2023-10-30T12:51:04.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.417+0000 7f0a177fe640 1 -- 172.21.15.8:0/2736366999 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180c47e0 con 0x7f0a18153c80 2023-10-30T12:51:04.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a177fe640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a18153c80 0x7f0a1815cf90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097c0 tx=0x7f0a0801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2382962816968749 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a16ffd640 1 -- 172.21.15.8:0/2736366999 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080a8970 con 0x7f0a18153c80 2023-10-30T12:51:04.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a16ffd640 1 -- 172.21.15.8:0/2736366999 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080a8ad0 con 0x7f0a18153c80 2023-10-30T12:51:04.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a16ffd640 1 -- 172.21.15.8:0/2736366999 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080a8e60 con 0x7f0a18153c80 2023-10-30T12:51:04.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/2736366999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a18153c80 msgr2=0x7f0a1815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a18153c80 0x7f0a1815cf90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097c0 tx=0x7f0a0801ec70 comp rx=0 tx=0).stop 2023-10-30T12:51:04.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/2736366999 shutdown_connections 2023-10-30T12:51:04.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a180c4a10 0x7f0a180c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/2736366999 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a18153c80 0x7f0a1815cf90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/2736366999 >> 172.21.15.8:0/2736366999 conn(0x7f0a180715c0 msgr2=0x7f0a180719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:04.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/2736366999 shutdown_connections 2023-10-30T12:51:04.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.421+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/2736366999 wait complete. 2023-10-30T12:51:04.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 Processor -- start 2023-10-30T12:51:04.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- start start 2023-10-30T12:51:04.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a18153c80 0x7f0a1815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a1815a3b0 0x7f0a1814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181ef280 con 0x7f0a180c4a10 2023-10-30T12:51:04.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f2c40 con 0x7f0a18153c80 2023-10-30T12:51:04.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181ef6a0 con 0x7f0a1815a3b0 2023-10-30T12:51:04.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37566/0 (socket says 172.21.15.8:37566) 2023-10-30T12:51:04.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 -- 172.21.15.8:0/1313385161 learned_addr learned my addr 172.21.15.8:0/1313385161 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:04.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a17fff640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a1815a3b0 0x7f0a1814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a0ffff640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a18153c80 0x7f0a1815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 -- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a1815a3b0 msgr2=0x7f0a1814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a1815a3b0 0x7f0a1814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 -- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a18153c80 msgr2=0x7f0a1815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a18153c80 0x7f0a1815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 -- 172.21.15.8:0/1313385161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08009420 con 0x7f0a180c4a10 2023-10-30T12:51:04.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a177fe640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a0801c8a0 tx=0x7f0a0801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080b0e10 con 0x7f0a180c4a10 2023-10-30T12:51:04.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080b9410 con 0x7f0a180c4a10 2023-10-30T12:51:04.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080b97c0 con 0x7f0a180c4a10 2023-10-30T12:51:04.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1814ff60 con 0x7f0a180c4a10 2023-10-30T12:51:04.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.425+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181eebc0 con 0x7f0a180c4a10 2023-10-30T12:51:04.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a181ef3c0 con 0x7f0a180c4a10 2023-10-30T12:51:04.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a080b7040 con 0x7f0a180c4a10 2023-10-30T12:51:04.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a14ff9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 0x7f09f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:04.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f0a08121c80 con 0x7f0a180c4a10 2023-10-30T12:51:04.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a0ffff640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 0x7f09f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:04.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.429+0000 7f0a0ffff640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 0x7f09f8068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a1815cac0 tx=0x7f0a00008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:04.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.433+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a181ef3c0 con 0x7f0a180c4a10 2023-10-30T12:51:04.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a181ef3c0 con 0x7f0a180c4a10 2023-10-30T12:51:04.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a14ff9640 1 -- 172.21.15.8:0/1313385161 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f0a181ef3c0 con 0x7f0a180c4a10 2023-10-30T12:51:04.628 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:04.629 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:02.460175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:resolve","state_seq":18,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:04.629 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 32 2023-10-30T12:51:04.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 msgr2=0x7f09f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 0x7f09f8068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a1815cac0 tx=0x7f0a00008040 comp rx=0 tx=0).stop 2023-10-30T12:51:04.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 msgr2=0x7f0a181f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:04.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a0801c8a0 tx=0x7f0a0801ba10 comp rx=0 tx=0).stop 2023-10-30T12:51:04.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 shutdown_connections 2023-10-30T12:51:04.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f09f80664a0 0x7f09f8068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a18153c80 0x7f0a1815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0a1815a3b0 0x7f0a1814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 --2- 172.21.15.8:0/1313385161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a180c4a10 0x7f0a181f2700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:04.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 >> 172.21.15.8:0/1313385161 conn(0x7f0a180715c0 msgr2=0x7f0a18157990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:04.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 shutdown_connections 2023-10-30T12:51:04.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:04.625+0000 7f0a1e0c9640 1 -- 172.21.15.8:0/1313385161 wait complete. 2023-10-30T12:51:04.649 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:06.650 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:06.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.801+0000 7f2d380bb640 1 Processor -- start 2023-10-30T12:51:06.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.801+0000 7f2d380bb640 1 -- start start 2023-10-30T12:51:06.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.801+0000 7f2d380bb640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2d30151c20 0x7f2d300c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.801+0000 7f2d380bb640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d3006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.801+0000 7f2d380bb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d300c5030 con 0x7f2d30151840 2023-10-30T12:51:06.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d380bb640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d30157830 con 0x7f2d30151c20 2023-10-30T12:51:06.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d380bb640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30157b10 con 0x7f2d300cb8b0 2023-10-30T12:51:06.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2d30151c20 0x7f2d300c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2d30151c20 0x7f2d300c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37582/0 (socket says 172.21.15.8:37582) 2023-10-30T12:51:06.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 -- 172.21.15.8:0/1622197895 learned_addr learned my addr 172.21.15.8:0/1622197895 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:06.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d36658640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d3006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 -- 172.21.15.8:0/1622197895 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 msgr2=0x7f2d3006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:06.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d3006b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:06.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 -- 172.21.15.8:0/1622197895 >> v1:172.21.15.46:6789/0 conn(0x7f2d30151840 legacy=0x7f2d30072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:06.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 -- 172.21.15.8:0/1622197895 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30157df0 con 0x7f2d30151c20 2023-10-30T12:51:06.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d35656640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151c20 0x7f2d300c49a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009870 tx=0x7f2d2001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e2319b20c80e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:06.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d34e55640 1 -- 172.21.15.8:0/1622197895 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200a8a30 con 0x7f2d30151c20 2023-10-30T12:51:06.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d34e55640 1 -- 172.21.15.8:0/1622197895 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d200a8b90 con 0x7f2d30151c20 2023-10-30T12:51:06.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.805+0000 7f2d34e55640 1 -- 172.21.15.8:0/1622197895 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200b0440 con 0x7f2d30151c20 2023-10-30T12:51:06.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- 172.21.15.8:0/1622197895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151c20 msgr2=0x7f2d300c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:06.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151c20 0x7f2d300c49a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009870 tx=0x7f2d2001ed30 comp rx=0 tx=0).stop 2023-10-30T12:51:06.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- 172.21.15.8:0/1622197895 shutdown_connections 2023-10-30T12:51:06.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d3006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:06.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1622197895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151c20 0x7f2d300c49a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:06.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- 172.21.15.8:0/1622197895 >> 172.21.15.8:0/1622197895 conn(0x7f2d300cac70 msgr2=0x7f2d300cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:06.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- 172.21.15.8:0/1622197895 shutdown_connections 2023-10-30T12:51:06.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- 172.21.15.8:0/1622197895 wait complete. 2023-10-30T12:51:06.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 Processor -- start 2023-10-30T12:51:06.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- start start 2023-10-30T12:51:06.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d301eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d30151c20 0x7f2d301465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301e2b30 con 0x7f2d30151840 2023-10-30T12:51:06.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301ef2e0 con 0x7f2d30151c20 2023-10-30T12:51:06.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d380bb640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301f3220 con 0x7f2d300cb8b0 2023-10-30T12:51:06.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d35656640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d35656640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37590/0 (socket says 172.21.15.8:37590) 2023-10-30T12:51:06.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d35656640 1 -- 172.21.15.8:0/1201573921 learned_addr learned my addr 172.21.15.8:0/1201573921 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:06.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d36658640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d30151c20 0x7f2d301465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d35e57640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d301eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.809+0000 7f2d35656640 1 -- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 msgr2=0x7f2d301eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:06.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35656640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d301eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:06.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35656640 1 -- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d30151c20 msgr2=0x7f2d301465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:06.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35656640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d30151c20 0x7f2d301465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:06.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35656640 1 -- 172.21.15.8:0/1201573921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200094d0 con 0x7f2d30151840 2023-10-30T12:51:06.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35656640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b3040 tx=0x7f2d2001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:06.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200be070 con 0x7f2d30151840 2023-10-30T12:51:06.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d200ba400 con 0x7f2d30151840 2023-10-30T12:51:06.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200ba7d0 con 0x7f2d30151840 2023-10-30T12:51:06.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d301f2d20 con 0x7f2d30151840 2023-10-30T12:51:06.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d301e2570 con 0x7f2d30151840 2023-10-30T12:51:06.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3004e170 con 0x7f2d30151840 2023-10-30T12:51:06.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2d200c3410 con 0x7f2d30151840 2023-10-30T12:51:06.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 0x7f2d14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:06.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f2d20121ba0 con 0x7f2d30151840 2023-10-30T12:51:06.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.813+0000 7f2d35e57640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 0x7f2d14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:06.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.817+0000 7f2d35e57640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 0x7f2d14068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000f70 tx=0x7f2d2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:06.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:06.817+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2d200eeea0 con 0x7f2d30151840 2023-10-30T12:51:07.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.005+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d30073e20 con 0x7f2d30151840 2023-10-30T12:51:07.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d267fc640 1 -- 172.21.15.8:0/1201573921 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f2d30073e20 con 0x7f2d30151840 2023-10-30T12:51:07.011 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:07.011 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:02.460175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:resolve","state_seq":18,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:07.011 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 32 2023-10-30T12:51:07.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 msgr2=0x7f2d14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 0x7f2d14068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000f70 tx=0x7f2d2c009040 comp rx=0 tx=0).stop 2023-10-30T12:51:07.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 msgr2=0x7f2d3006b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:07.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b3040 tx=0x7f2d2001bad0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 shutdown_connections 2023-10-30T12:51:07.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2d140664a0 0x7f2d14068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d30151c20 0x7f2d301465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d30151840 0x7f2d3006b6d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 --2- 172.21.15.8:0/1201573921 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2d300cb8b0 0x7f2d301eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:07.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 >> 172.21.15.8:0/1201573921 conn(0x7f2d300cac70 msgr2=0x7f2d30072000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:07.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 shutdown_connections 2023-10-30T12:51:07.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:07.009+0000 7f2d380bb640 1 -- 172.21.15.8:0/1201573921 wait complete. 2023-10-30T12:51:07.031 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:09.033 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:09.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 Processor -- start 2023-10-30T12:51:09.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 -- start start 2023-10-30T12:51:09.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc774150830 0x7fc77406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7740c2310 con 0x7fc774070520 2023-10-30T12:51:09.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc774156710 con 0x7fc7740cb7d0 2023-10-30T12:51:09.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc77a246640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7740c4360 con 0x7fc774150830 2023-10-30T12:51:09.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc773fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fc774070520 0x7fc774160f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60322/0 (socket says 172.21.15.8:60322) 2023-10-30T12:51:09.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc773fff640 1 -- 172.21.15.8:0/1598313791 learned_addr learned my addr 172.21.15.8:0/1598313791 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:09.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc7737fe640 1 --2- 172.21.15.8:0/1598313791 >> v2:172.21.15.46:3300/0 conn(0x7fc774150830 0x7fc77406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126098706 0 0) 0x7fc7740c2310 con 0x7fc774070520 2023-10-30T12:51:09.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.185+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc75c005740 con 0x7fc774070520 2023-10-30T12:51:09.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1064795007 0 0) 0x7fc75c005740 con 0x7fc774070520 2023-10-30T12:51:09.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 >> v2:172.21.15.46:3300/0 conn(0x7fc774150830 msgr2=0x7fc77406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:09.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 --2- 172.21.15.8:0/1598313791 >> v2:172.21.15.46:3300/0 conn(0x7fc774150830 0x7fc77406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 >> v1:172.21.15.46:6789/0 conn(0x7fc7740cb7d0 legacy=0x7fc7740c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7741569b0 con 0x7fc774070520 2023-10-30T12:51:09.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fc76c001a20 con 0x7fc774070520 2023-10-30T12:51:09.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 >> v1:172.21.15.8:6789/0 conn(0x7fc774070520 legacy=0x7fc774160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 0x7fc75c005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc75c006590 0x7fc75c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c007260 0x7fc75c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740c4360 con 0x7fc75c005b90 2023-10-30T12:51:09.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c00e110 con 0x7fc75c007260 2023-10-30T12:51:09.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c00e2d0 con 0x7fc75c006590 2023-10-30T12:51:09.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 0x7fc75c005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc773fff640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c007260 0x7fc75c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc772ffd640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc75c006590 0x7fc75c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 -- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc75c006590 msgr2=0x7fc75c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:09.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc75c006590 0x7fc75c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 -- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c007260 msgr2=0x7fc75c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:09.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c007260 0x7fc75c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 -- 172.21.15.8:0/1598313791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c005740 con 0x7fc75c005b90 2023-10-30T12:51:09.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc773fff640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c007260 0x7fc75c00dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:09.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc773fff640 1 -- 172.21.15.8:0/1598313791 reap_dead start 2023-10-30T12:51:09.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.189+0000 7fc7737fe640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 0x7fc75c005f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ae0 tx=0x7fc76401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387a6c9178f293ca server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a5640 con 0x7fc75c005b90 2023-10-30T12:51:09.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7640a57a0 con 0x7fc75c005b90 2023-10-30T12:51:09.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc7727fc640 1 -- 172.21.15.8:0/1598313791 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640a5b50 con 0x7fc75c005b90 2023-10-30T12:51:09.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 msgr2=0x7fc75c005f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 0x7fc75c005f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ae0 tx=0x7fc76401ca50 comp rx=0 tx=0).stop 2023-10-30T12:51:09.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- 172.21.15.8:0/1598313791 shutdown_connections 2023-10-30T12:51:09.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 --2- 172.21.15.8:0/1598313791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc75c005b90 0x7fc75c005f70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- 172.21.15.8:0/1598313791 >> 172.21.15.8:0/1598313791 conn(0x7fc7740cac70 msgr2=0x7fc7740cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:09.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- 172.21.15.8:0/1598313791 shutdown_connections 2023-10-30T12:51:09.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- 172.21.15.8:0/1598313791 wait complete. 2023-10-30T12:51:09.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 Processor -- start 2023-10-30T12:51:09.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- start start 2023-10-30T12:51:09.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c005b90 0x7fc7741ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc77414cfa0 0x7fc77414fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77414d3e0 con 0x7fc774154f20 2023-10-30T12:51:09.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774154a90 con 0x7fc75c005b90 2023-10-30T12:51:09.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc77a246640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741558e0 con 0x7fc77414cfa0 2023-10-30T12:51:09.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc773fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.193+0000 7fc773fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55498/0 (socket says 172.21.15.8:55498) 2023-10-30T12:51:09.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 -- 172.21.15.8:0/752777863 learned_addr learned my addr 172.21.15.8:0/752777863 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:09.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc7737fe640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c005b90 0x7fc7741ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc772ffd640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc77414cfa0 0x7fc77414fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 -- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc77414cfa0 msgr2=0x7fc77414fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc77414cfa0 0x7fc77414fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 -- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c005b90 msgr2=0x7fc7741ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c005b90 0x7fc7741ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 -- 172.21.15.8:0/752777863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77414de80 con 0x7fc774154f20 2023-10-30T12:51:09.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc773fff640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc774072e90 tx=0x7fc76c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c00ecc0 con 0x7fc774154f20 2023-10-30T12:51:09.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c00ee20 con 0x7fc774154f20 2023-10-30T12:51:09.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c014670 con 0x7fc774154f20 2023-10-30T12:51:09.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77414d5e0 con 0x7fc774154f20 2023-10-30T12:51:09.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77414dba0 con 0x7fc774154f20 2023-10-30T12:51:09.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc774066510 con 0x7fc774154f20 2023-10-30T12:51:09.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc76c0147d0 con 0x7fc774154f20 2023-10-30T12:51:09.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 0x7fc7600689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:09.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fc76c083cd0 con 0x7fc774154f20 2023-10-30T12:51:09.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.197+0000 7fc7737fe640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 0x7fc7600689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:09.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.201+0000 7fc7737fe640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 0x7fc7600689e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af000 tx=0x7fc7640046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:09.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.205+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc76c051100 con 0x7fc774154f20 2023-10-30T12:51:09.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc77404e170 con 0x7fc774154f20 2023-10-30T12:51:09.400 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc75bfff640 1 -- 172.21.15.8:0/752777863 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5733 (secure 0 0 0) 0x7fc76c054720 con 0x7fc774154f20 2023-10-30T12:51:09.401 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:09.401 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:08.240442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:09.401 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 33 2023-10-30T12:51:09.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 msgr2=0x7fc7600689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 0x7fc7600689e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af000 tx=0x7fc7640046d0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 msgr2=0x7fc7741553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:09.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.397+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc774072e90 tx=0x7fc76c00e530 comp rx=0 tx=0).stop 2023-10-30T12:51:09.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 shutdown_connections 2023-10-30T12:51:09.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc760066520 0x7fc7600689e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc774154f20 0x7fc7741553a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc77414cfa0 0x7fc77414fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 --2- 172.21.15.8:0/752777863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc75c005b90 0x7fc7741ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:09.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 >> 172.21.15.8:0/752777863 conn(0x7fc7740cac70 msgr2=0x7fc7741e0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:09.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 shutdown_connections 2023-10-30T12:51:09.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:09.401+0000 7fc77a246640 1 -- 172.21.15.8:0/752777863 wait complete. 2023-10-30T12:51:09.422 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:11.423 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:11.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.573+0000 7f9ea453e640 1 Processor -- start 2023-10-30T12:51:11.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.573+0000 7f9ea453e640 1 -- start start 2023-10-30T12:51:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea453e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e9c065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea453e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c06e390 con 0x7f9e9c070c30 2023-10-30T12:51:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea453e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0660e0 con 0x7f9e9c159970 2023-10-30T12:51:11.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea453e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c072390 con 0x7f9e9c153c10 2023-10-30T12:51:11.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f9e9c070c30 0x7f9e9c071070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60326/0 (socket says 172.21.15.8:60326) 2023-10-30T12:51:11.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 -- 172.21.15.8:0/1797150313 learned_addr learned my addr 172.21.15.8:0/1797150313 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea22da640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e9c065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150659175 0 0) 0x7f9e9c06e390 con 0x7f9e9c070c30 2023-10-30T12:51:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e88003530 con 0x7f9e9c070c30 2023-10-30T12:51:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270487678 0 0) 0x7f9e88003530 con 0x7f9e9c070c30 2023-10-30T12:51:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 msgr2=0x7f9e9c065ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:11.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e9c065ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 >> v1:172.21.15.46:6789/0 conn(0x7f9e9c159970 legacy=0x7f9e9c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0bec10 con 0x7f9e9c070c30 2023-10-30T12:51:11.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f9e8c001680 con 0x7f9e9c070c30 2023-10-30T12:51:11.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 >> v1:172.21.15.8:6789/0 conn(0x7f9e9c070c30 legacy=0x7f9e9c071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e88004d10 0x7f9e88005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c072390 con 0x7f9e88003ef0 2023-10-30T12:51:11.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8800bad0 con 0x7f9e88004d10 2023-10-30T12:51:11.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8800bc90 con 0x7f9e9c153c10 2023-10-30T12:51:11.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea22da640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea2adb640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e88004d10 0x7f9e88005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 -- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 msgr2=0x7f9e880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c153c10 0x7f9e880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 -- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e88004d10 msgr2=0x7f9e88005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e88004d10 0x7f9e88005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.577+0000 7f9ea1ad9640 1 -- 172.21.15.8:0/1797150313 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88003530 con 0x7f9e88003ef0 2023-10-30T12:51:11.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e880047d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c110 tx=0x7f9e8c01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0b7351a1556cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:11.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c005ae0 con 0x7f9e88003ef0 2023-10-30T12:51:11.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c005c40 con 0x7f9e88003ef0 2023-10-30T12:51:11.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea12d8640 1 -- 172.21.15.8:0/1797150313 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0a5860 con 0x7f9e88003ef0 2023-10-30T12:51:11.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 -- 172.21.15.8:0/1797150313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 msgr2=0x7f9e880047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1797150313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e880047d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c110 tx=0x7f9e8c01cb60 comp rx=0 tx=0).stop 2023-10-30T12:51:11.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea2adb640 1 -- 172.21.15.8:0/1797150313 reap_dead start 2023-10-30T12:51:11.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 -- 172.21.15.8:0/1797150313 shutdown_connections 2023-10-30T12:51:11.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 -- 172.21.15.8:0/1797150313 >> 172.21.15.8:0/1797150313 conn(0x7f9e9c0c7f60 msgr2=0x7f9e9c0c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:11.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 -- 172.21.15.8:0/1797150313 shutdown_connections 2023-10-30T12:51:11.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.581+0000 7f9ea453e640 1 -- 172.21.15.8:0/1797150313 wait complete. 2023-10-30T12:51:11.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 Processor -- start 2023-10-30T12:51:11.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- start start 2023-10-30T12:51:11.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 0x7f9e9c0be220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e9c0c97a0 0x7f9e9c14a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0c3460 con 0x7f9e88003ef0 2023-10-30T12:51:11.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c1f2dc0 con 0x7f9e9c0c97a0 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0c9d00 con 0x7f9e9c0c2870 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55526/0 (socket says 172.21.15.8:55526) 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 -- 172.21.15.8:0/1570275679 learned_addr learned my addr 172.21.15.8:0/1570275679 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 0x7f9e9c0be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea2adb640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e9c0c97a0 0x7f9e9c14a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 -- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 msgr2=0x7f9e9c0be220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 0x7f9e9c0be220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 -- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e9c0c97a0 msgr2=0x7f9e9c14a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e9c0c97a0 0x7f9e9c14a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 -- 172.21.15.8:0/1570275679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0c55e0 con 0x7f9e88003ef0 2023-10-30T12:51:11.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 0x7f9e9c0be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:11.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea22da640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e98011a30 tx=0x7f9e98011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:11.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800fb50 con 0x7f9e88003ef0 2023-10-30T12:51:11.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9800fcb0 con 0x7f9e88003ef0 2023-10-30T12:51:11.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98014440 con 0x7f9e88003ef0 2023-10-30T12:51:11.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c1f2140 con 0x7f9e88003ef0 2023-10-30T12:51:11.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c1f2700 con 0x7f9e88003ef0 2023-10-30T12:51:11.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.585+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c04e170 con 0x7f9e88003ef0 2023-10-30T12:51:11.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.589+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e980145a0 con 0x7f9e88003ef0 2023-10-30T12:51:11.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.589+0000 7f9e92ffd640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 0x7f9e80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:11.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.589+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f9e980847e0 con 0x7f9e88003ef0 2023-10-30T12:51:11.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.589+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 0x7f9e80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:11.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.589+0000 7f9ea1ad9640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 0x7f9e80068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0ab000 tx=0x7f9e8c0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:11.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.593+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e98051ba0 con 0x7f9e88003ef0 2023-10-30T12:51:11.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e9c0c5880 con 0x7f9e88003ef0 2023-10-30T12:51:11.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9e92ffd640 1 -- 172.21.15.8:0/1570275679 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f9e980551c0 con 0x7f9e88003ef0 2023-10-30T12:51:11.784 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:11.785 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:08.240442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 33 2023-10-30T12:51:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 msgr2=0x7f9e80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 0x7f9e80068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0ab000 tx=0x7f9e8c0a7070 comp rx=0 tx=0).stop 2023-10-30T12:51:11.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 msgr2=0x7f9e9c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e98011a30 tx=0x7f9e98011ef0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 shutdown_connections 2023-10-30T12:51:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9e800664a0 0x7f9e80068960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e9c0c97a0 0x7f9e9c14a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9e9c0c2870 0x7f9e9c0be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 --2- 172.21.15.8:0/1570275679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e88003ef0 0x7f9e9c0c50a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 >> 172.21.15.8:0/1570275679 conn(0x7f9e9c0c7f60 msgr2=0x7f9e9c14b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 shutdown_connections 2023-10-30T12:51:11.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:11.781+0000 7f9ea453e640 1 -- 172.21.15.8:0/1570275679 wait complete. 2023-10-30T12:51:11.806 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 33, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 33, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 33 }, "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:51:08.240442+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 5079, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:51:13.807 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:13.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.957+0000 7f9742f72640 1 Processor -- start 2023-10-30T12:51:13.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.957+0000 7f9742f72640 1 -- start start 2023-10-30T12:51:13.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9742f72640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f973c150830 0x7f973c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9742f72640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f973c070520 0x7f973c070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9742f72640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0c2310 con 0x7f973c0cb7d0 2023-10-30T12:51:13.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9742f72640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c156710 con 0x7f973c150830 2023-10-30T12:51:13.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9742f72640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0c4360 con 0x7f973c070520 2023-10-30T12:51:13.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f973c150830 0x7f973c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f973bfff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f973c0cb7d0 0x7f973c0c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60332/0 (socket says 172.21.15.8:60332) 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f973c150830 0x7f973c06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55528/0 (socket says 172.21.15.8:55528) 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f973bfff640 1 -- 172.21.15.8:0/2716670262 learned_addr learned my addr 172.21.15.8:0/2716670262 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f974150f640 1 --2- 172.21.15.8:0/2716670262 >> v2:172.21.15.46:3300/0 conn(0x7f973c070520 0x7f973c070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 -- 172.21.15.8:0/2716670262 >> v2:172.21.15.46:3300/0 conn(0x7f973c070520 msgr2=0x7f973c070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 --2- 172.21.15.8:0/2716670262 >> v2:172.21.15.46:3300/0 conn(0x7f973c070520 0x7f973c070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:13.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 -- 172.21.15.8:0/2716670262 >> v1:172.21.15.8:6789/0 conn(0x7f973c0cb7d0 legacy=0x7f973c0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:13.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 -- 172.21.15.8:0/2716670262 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c1569b0 con 0x7f973c150830 2023-10-30T12:51:13.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f973b7fe640 1 -- 172.21.15.8:0/2716670262 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442923927 0 0) 0x7f973c0c2310 con 0x7f973c0cb7d0 2023-10-30T12:51:13.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.961+0000 7f9740d0e640 1 --2- 172.21.15.8:0/2716670262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c06b500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f973c066e10 tx=0x7f973001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fa5b601f2127b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:13.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f973b7fe640 1 -- 172.21.15.8:0/2716670262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300a8950 con 0x7f973c150830 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f973b7fe640 1 -- 172.21.15.8:0/2716670262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300a8ab0 con 0x7f973c150830 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f973b7fe640 1 -- 172.21.15.8:0/2716670262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300a8e40 con 0x7f973c150830 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- 172.21.15.8:0/2716670262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 msgr2=0x7f973c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- 172.21.15.8:0/2716670262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c06b500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f973c066e10 tx=0x7f973001ec50 comp rx=0 tx=0).stop 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- 172.21.15.8:0/2716670262 shutdown_connections 2023-10-30T12:51:13.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- 172.21.15.8:0/2716670262 >> v2:172.21.15.46:3300/0 conn(0x7f973c070520 0x7f973c070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:13.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- 172.21.15.8:0/2716670262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c06b500 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:13.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- 172.21.15.8:0/2716670262 >> 172.21.15.8:0/2716670262 conn(0x7f973c0cac70 msgr2=0x7f973c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:13.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- 172.21.15.8:0/2716670262 shutdown_connections 2023-10-30T12:51:13.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- 172.21.15.8:0/2716670262 wait complete. 2023-10-30T12:51:13.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 Processor -- start 2023-10-30T12:51:13.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- start start 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973c070520 0x7f973c1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 0x7f973c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15a180 con 0x7f973c150830 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c9c30 con 0x7f973c070520 2023-10-30T12:51:13.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.965+0000 7f9742f72640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c15a440 con 0x7f973c0cb7d0 2023-10-30T12:51:13.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55534/0 (socket says 172.21.15.8:55534) 2023-10-30T12:51:13.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f9740d0e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973c070520 0x7f973c1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 -- 172.21.15.8:0/129111255 learned_addr learned my addr 172.21.15.8:0/129111255 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:13.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f973bfff640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 0x7f973c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 -- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 msgr2=0x7f973c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:13.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 0x7f973c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:13.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 -- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973c070520 msgr2=0x7f973c1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:13.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973c070520 0x7f973c1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:13.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 -- 172.21.15.8:0/129111255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730009420 con 0x7f973c150830 2023-10-30T12:51:13.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f973bfff640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 0x7f973c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:13.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f974150f640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9734007e40 tx=0x7f973400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:13.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734014620 con 0x7f973c150830 2023-10-30T12:51:13.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400ee90 con 0x7f973c150830 2023-10-30T12:51:13.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c15a700 con 0x7f973c150830 2023-10-30T12:51:13.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340149e0 con 0x7f973c150830 2023-10-30T12:51:13.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0c98c0 con 0x7f973c150830 2023-10-30T12:51:13.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c0c9430 con 0x7f973c150830 2023-10-30T12:51:13.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9734014b40 con 0x7f973c150830 2023-10-30T12:51:13.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 0x7f9720068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:13.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.969+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f97340848f0 con 0x7f973c150830 2023-10-30T12:51:13.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.973+0000 7f9740d0e640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 0x7f9720068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:13.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.973+0000 7f9740d0e640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 0x7f9720068990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f97300b2000 tx=0x7f9730008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:13.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:13.977+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9734051bf0 con 0x7f973c150830 2023-10-30T12:51:14.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f973c04e170 con 0x7f973c150830 2023-10-30T12:51:14.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f97397fa640 1 -- 172.21.15.8:0/129111255 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5730 (secure 0 0 0) 0x7f9734055210 con 0x7f973c150830 2023-10-30T12:51:14.169 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:14.169 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:13.258369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:14.169 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 34 2023-10-30T12:51:14.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 msgr2=0x7f9720068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 0x7f9720068990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f97300b2000 tx=0x7f9730008040 comp rx=0 tx=0).stop 2023-10-30T12:51:14.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 msgr2=0x7f973c158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:14.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9734007e40 tx=0x7f973400e4b0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 shutdown_connections 2023-10-30T12:51:14.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f97200664d0 0x7f9720068990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f973c150830 0x7f973c158390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f973c0cb7d0 0x7f973c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 --2- 172.21.15.8:0/129111255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f973c070520 0x7f973c1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:14.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.165+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 >> 172.21.15.8:0/129111255 conn(0x7f973c0cac70 msgr2=0x7f973c0736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:14.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.169+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 shutdown_connections 2023-10-30T12:51:14.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:14.169+0000 7f9742f72640 1 -- 172.21.15.8:0/129111255 wait complete. 2023-10-30T12:51:14.189 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:15.094 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:15.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:16.190 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:16.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.341+0000 7f8463394640 1 Processor -- start 2023-10-30T12:51:16.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.341+0000 7f8463394640 1 -- start start 2023-10-30T12:51:16.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8463394640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 0x7f845c0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8463394640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8463394640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c0c5030 con 0x7f845c0cb8b0 2023-10-30T12:51:16.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8463394640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c157830 con 0x7f845c151840 2023-10-30T12:51:16.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8463394640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c157b10 con 0x7f845c151c20 2023-10-30T12:51:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8461130640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 0x7f845c0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:46844/0 (socket says 172.21.15.8:46844) 2023-10-30T12:51:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 -- 172.21.15.8:0/4201312096 learned_addr learned my addr 172.21.15.8:0/4201312096 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 -- 172.21.15.8:0/4201312096 >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 msgr2=0x7f845c0c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 --2- 172.21.15.8:0/4201312096 >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 0x7f845c0c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 -- 172.21.15.8:0/4201312096 >> v1:172.21.15.46:6789/0 conn(0x7f845c0cb8b0 legacy=0x7f845c06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f846092f640 1 -- 172.21.15.8:0/4201312096 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c157df0 con 0x7f845c151c20 2023-10-30T12:51:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.345+0000 7f8461130640 1 --2- 172.21.15.8:0/4201312096 >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 0x7f845c0c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:51:16.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f846092f640 1 --2- 172.21.15.8:0/4201312096 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f844c009870 tx=0x7f844c01ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7c88b401f65c19 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:16.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8453fff640 1 -- 172.21.15.8:0/4201312096 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0a8a30 con 0x7f845c151c20 2023-10-30T12:51:16.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8453fff640 1 -- 172.21.15.8:0/4201312096 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c0a8b90 con 0x7f845c151c20 2023-10-30T12:51:16.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8453fff640 1 -- 172.21.15.8:0/4201312096 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b0440 con 0x7f845c151c20 2023-10-30T12:51:16.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 -- 172.21.15.8:0/4201312096 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 msgr2=0x7f845c072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 --2- 172.21.15.8:0/4201312096 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f844c009870 tx=0x7f844c01ed30 comp rx=0 tx=0).stop 2023-10-30T12:51:16.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 -- 172.21.15.8:0/4201312096 shutdown_connections 2023-10-30T12:51:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 --2- 172.21.15.8:0/4201312096 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c072000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 --2- 172.21.15.8:0/4201312096 >> v2:172.21.15.46:3300/0 conn(0x7f845c151840 0x7f845c0c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 -- 172.21.15.8:0/4201312096 >> 172.21.15.8:0/4201312096 conn(0x7f845c0cac70 msgr2=0x7f845c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 -- 172.21.15.8:0/4201312096 shutdown_connections 2023-10-30T12:51:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.349+0000 7f8463394640 1 -- 172.21.15.8:0/4201312096 wait complete. 2023-10-30T12:51:16.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 Processor -- start 2023-10-30T12:51:16.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- start start 2023-10-30T12:51:16.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f845c0cb8b0 0x7f845c1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c073d10 con 0x7f845c151840 2023-10-30T12:51:16.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c153910 con 0x7f845c0cb8b0 2023-10-30T12:51:16.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c157090 con 0x7f845c151c20 2023-10-30T12:51:16.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55542/0 (socket says 172.21.15.8:55542) 2023-10-30T12:51:16.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 -- 172.21.15.8:0/3496773030 learned_addr learned my addr 172.21.15.8:0/3496773030 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:16.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8461130640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f845c0cb8b0 0x7f845c1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8461931640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 -- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 msgr2=0x7f845c1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 -- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f845c0cb8b0 msgr2=0x7f845c1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f845c0cb8b0 0x7f845c1f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 -- 172.21.15.8:0/3496773030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009420 con 0x7f845c151840 2023-10-30T12:51:16.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f846092f640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b3040 tx=0x7f844c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:16.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b9620 con 0x7f845c151840 2023-10-30T12:51:16.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c0b97a0 con 0x7f845c151840 2023-10-30T12:51:16.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b9b50 con 0x7f845c151840 2023-10-30T12:51:16.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c0094d0 con 0x7f845c151840 2023-10-30T12:51:16.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c1575c0 con 0x7f845c151840 2023-10-30T12:51:16.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.353+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c066510 con 0x7f845c151840 2023-10-30T12:51:16.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.357+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f844c0b7070 con 0x7f845c151840 2023-10-30T12:51:16.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.357+0000 7f8451ffb640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 0x7f8440068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:16.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.357+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f844c121ea0 con 0x7f845c151840 2023-10-30T12:51:16.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.357+0000 7f8461130640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 0x7f8440068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:16.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.357+0000 7f8461130640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 0x7f8440068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8458009c80 tx=0x7f8458008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:16.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.361+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f844c0ef1a0 con 0x7f845c151840 2023-10-30T12:51:16.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.553+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f845c04e170 con 0x7f845c151840 2023-10-30T12:51:16.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.553+0000 7f8451ffb640 1 -- 172.21.15.8:0/3496773030 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5730 (secure 0 0 0) 0x7f844c0f27c0 con 0x7f845c151840 2023-10-30T12:51:16.557 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:16.557 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:13.258369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:16.558 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 34 2023-10-30T12:51:16.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 msgr2=0x7f8440068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 0x7f8440068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8458009c80 tx=0x7f8458008040 comp rx=0 tx=0).stop 2023-10-30T12:51:16.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 msgr2=0x7f845c153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:16.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b3040 tx=0x7f844c01bad0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 shutdown_connections 2023-10-30T12:51:16.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84400664a0 0x7f8440068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f845c151c20 0x7f845c1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f845c151840 0x7f845c153da0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 --2- 172.21.15.8:0/3496773030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f845c0cb8b0 0x7f845c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:16.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 >> 172.21.15.8:0/3496773030 conn(0x7f845c0cac70 msgr2=0x7f845c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:16.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 shutdown_connections 2023-10-30T12:51:16.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:16.557+0000 7f8463394640 1 -- 172.21.15.8:0/3496773030 wait complete. 2023-10-30T12:51:16.578 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:18.579 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.729+0000 7fdf3c86b640 1 Processor -- start 2023-10-30T12:51:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.729+0000 7fdf3c86b640 1 -- start start 2023-10-30T12:51:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf381537f0 0x7fdf380701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf3815a2e0 0x7fdf380ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf380cd4a0 0x7fdf3806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380c3ab0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380706e0 con 0x7fdf380cd4a0 2023-10-30T12:51:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3c86b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c2aa0 con 0x7fdf381537f0 2023-10-30T12:51:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf3815a2e0 0x7fdf380ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf3815a2e0 0x7fdf380ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55556/0 (socket says 172.21.15.8:55556) 2023-10-30T12:51:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 -- 172.21.15.8:0/2678920516 learned_addr learned my addr 172.21.15.8:0/2678920516 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf3659c640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf381537f0 0x7fdf380701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf36d9d640 1 --2- 172.21.15.8:0/2678920516 >> v2:172.21.15.46:3300/0 conn(0x7fdf380cd4a0 0x7fdf3806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 -- 172.21.15.8:0/2678920516 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf381537f0 msgr2=0x7fdf380701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf381537f0 0x7fdf380701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 -- 172.21.15.8:0/2678920516 >> v2:172.21.15.46:3300/0 conn(0x7fdf380cd4a0 msgr2=0x7fdf3806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:18.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 --2- 172.21.15.8:0/2678920516 >> v2:172.21.15.46:3300/0 conn(0x7fdf380cd4a0 0x7fdf3806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.733+0000 7fdf35d9b640 1 -- 172.21.15.8:0/2678920516 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380708e0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf35d9b640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380ccf60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf38071f60 tx=0x7fdf2c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5edb19918b6d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:18.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3559a640 1 -- 172.21.15.8:0/2678920516 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0a89b0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3559a640 1 -- 172.21.15.8:0/2678920516 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c0a8b10 con 0x7fdf3815a2e0 2023-10-30T12:51:18.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3559a640 1 -- 172.21.15.8:0/2678920516 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0a8ea0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 -- 172.21.15.8:0/2678920516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 msgr2=0x7fdf380ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:18.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380ccf60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf38071f60 tx=0x7fdf2c01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 -- 172.21.15.8:0/2678920516 shutdown_connections 2023-10-30T12:51:18.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/2678920516 >> v2:172.21.15.46:3300/0 conn(0x7fdf380cd4a0 0x7fdf3806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380ccf60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/2678920516 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf381537f0 0x7fdf380701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 -- 172.21.15.8:0/2678920516 >> 172.21.15.8:0/2678920516 conn(0x7fdf38071fb0 msgr2=0x7fdf380723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:18.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 -- 172.21.15.8:0/2678920516 shutdown_connections 2023-10-30T12:51:18.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 -- 172.21.15.8:0/2678920516 wait complete. 2023-10-30T12:51:18.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.737+0000 7fdf3c86b640 1 Processor -- start 2023-10-30T12:51:18.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- start start 2023-10-30T12:51:18.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf380cd4a0 0x7fdf380c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf381537f0 0x7fdf380ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c47c0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c8770 con 0x7fdf381537f0 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c8a50 con 0x7fdf380cd4a0 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55558/0 (socket says 172.21.15.8:55558) 2023-10-30T12:51:18.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 -- 172.21.15.8:0/3200971270 learned_addr learned my addr 172.21.15.8:0/3200971270 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:18.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3659c640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf380cd4a0 0x7fdf380c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf35d9b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf381537f0 0x7fdf380ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 -- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf380cd4a0 msgr2=0x7fdf380c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:18.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf380cd4a0 0x7fdf380c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 -- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf381537f0 msgr2=0x7fdf380ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf381537f0 0x7fdf380ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 -- 172.21.15.8:0/3200971270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0094d0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf36d9d640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007d20 tx=0x7fdf3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30014420 con 0x7fdf3815a2e0 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf300145a0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30014950 con 0x7fdf3815a2e0 2023-10-30T12:51:18.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380c8d30 con 0x7fdf3815a2e0 2023-10-30T12:51:18.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380c92f0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.741+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3804e170 con 0x7fdf3815a2e0 2023-10-30T12:51:18.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.745+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf30014ab0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.745+0000 7fdf26ffd640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 0x7fdf18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:18.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.745+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fdf30084810 con 0x7fdf3815a2e0 2023-10-30T12:51:18.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.745+0000 7fdf3659c640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 0x7fdf18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:18.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.745+0000 7fdf3659c640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 0x7fdf18068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf28007a60 tx=0x7fdf28008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:18.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.749+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf30051b00 con 0x7fdf3815a2e0 2023-10-30T12:51:18.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf3806c0b0 con 0x7fdf3815a2e0 2023-10-30T12:51:18.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf26ffd640 1 -- 172.21.15.8:0/3200971270 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5730 (secure 0 0 0) 0x7fdf30055120 con 0x7fdf3815a2e0 2023-10-30T12:51:18.940 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:18.940 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:13.258369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:18.941 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 34 2023-10-30T12:51:18.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 msgr2=0x7fdf18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:18.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 0x7fdf18068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf28007a60 tx=0x7fdf28008040 comp rx=0 tx=0).stop 2023-10-30T12:51:18.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 msgr2=0x7fdf380c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:18.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007d20 tx=0x7fdf3000e4b0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 shutdown_connections 2023-10-30T12:51:18.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdf180664a0 0x7fdf18068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf381537f0 0x7fdf380ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf3815a2e0 0x7fdf380c5350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 --2- 172.21.15.8:0/3200971270 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf380cd4a0 0x7fdf380c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 >> 172.21.15.8:0/3200971270 conn(0x7fdf38071fb0 msgr2=0x7fdf380733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 shutdown_connections 2023-10-30T12:51:18.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:18.937+0000 7fdf3c86b640 1 -- 172.21.15.8:0/3200971270 wait complete. 2023-10-30T12:51:18.961 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:20.962 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:21.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 Processor -- start 2023-10-30T12:51:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 -- start start 2023-10-30T12:51:21.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3b14153c80 0x7f3b1415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b140c42a0 0x7f3b140ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b14159f20 con 0x7f3b140c4a10 2023-10-30T12:51:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1415dec0 con 0x7f3b14153c80 2023-10-30T12:51:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1c3d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b141599a0 con 0x7f3b140c42a0 2023-10-30T12:51:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a16f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3b14153c80 0x7f3b1415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1996e640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3b140c4a10 0x7f3b1415a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55244/0 (socket says 172.21.15.8:55244) 2023-10-30T12:51:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1996e640 1 -- 172.21.15.8:0/127382631 learned_addr learned my addr 172.21.15.8:0/127382631 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a970640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b140c42a0 0x7f3b140ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a16f640 1 -- 172.21.15.8:0/127382631 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b140c42a0 msgr2=0x7f3b140ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b140c42a0 0x7f3b140ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a16f640 1 -- 172.21.15.8:0/127382631 >> v1:172.21.15.8:6789/0 conn(0x7f3b140c4a10 legacy=0x7f3b1415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1a16f640 1 -- 172.21.15.8:0/127382631 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1415d620 con 0x7f3b14153c80 2023-10-30T12:51:21.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.113+0000 7f3b1916d640 1 -- 172.21.15.8:0/127382631 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292469681 0 0) 0x7f3b14159f20 con 0x7f3b140c4a10 2023-10-30T12:51:21.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b14153c80 0x7f3b1415cf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3b100097c0 tx=0x7f3b1001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e98d2da2c37dfe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:21.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1916d640 1 -- 172.21.15.8:0/127382631 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100a8970 con 0x7f3b14153c80 2023-10-30T12:51:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1916d640 1 -- 172.21.15.8:0/127382631 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b100a8ad0 con 0x7f3b14153c80 2023-10-30T12:51:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1916d640 1 -- 172.21.15.8:0/127382631 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100a8e60 con 0x7f3b14153c80 2023-10-30T12:51:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/127382631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b14153c80 msgr2=0x7f3b1415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b14153c80 0x7f3b1415cf90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3b100097c0 tx=0x7f3b1001ec70 comp rx=0 tx=0).stop 2023-10-30T12:51:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/127382631 shutdown_connections 2023-10-30T12:51:21.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b140c42a0 0x7f3b140ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/127382631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b14153c80 0x7f3b1415cf90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/127382631 >> 172.21.15.8:0/127382631 conn(0x7f3b140715c0 msgr2=0x7f3b140719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:21.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/127382631 shutdown_connections 2023-10-30T12:51:21.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.117+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/127382631 wait complete. 2023-10-30T12:51:21.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 Processor -- start 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- start start 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 0x7f3b14154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 0x7f3b14155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14156620 con 0x7f3b140c4a10 2023-10-30T12:51:21.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140ceac0 con 0x7f3b140c42a0 2023-10-30T12:51:21.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14154850 con 0x7f3b14153c80 2023-10-30T12:51:21.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53430/0 (socket says 172.21.15.8:53430) 2023-10-30T12:51:21.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 -- 172.21.15.8:0/345271169 learned_addr learned my addr 172.21.15.8:0/345271169 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:21.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 0x7f3b14154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1a970640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 0x7f3b14155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 -- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 msgr2=0x7f3b14155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 0x7f3b14155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 -- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 msgr2=0x7f3b14154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 0x7f3b14154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 -- 172.21.15.8:0/345271169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10009420 con 0x7f3b140c4a10 2023-10-30T12:51:21.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1a970640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 0x7f3b14155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:21.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 0x7f3b14154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:21.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1996e640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400c4e0 tx=0x7f3b0400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:21.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b040074e0 con 0x7f3b140c4a10 2023-10-30T12:51:21.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04007660 con 0x7f3b140c4a10 2023-10-30T12:51:21.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04004a30 con 0x7f3b140c4a10 2023-10-30T12:51:21.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b141f29f0 con 0x7f3b140c4a10 2023-10-30T12:51:21.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.121+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b141f2fa0 con 0x7f3b140c4a10 2023-10-30T12:51:21.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1404e170 con 0x7f3b140c4a10 2023-10-30T12:51:21.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b04004b90 con 0x7f3b140c4a10 2023-10-30T12:51:21.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b0affd640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:21.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f3b04083390 con 0x7f3b140c4a10 2023-10-30T12:51:21.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:21.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.125+0000 7f3b1a16f640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 0x7f3af8068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b2000 tx=0x7f3b10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:21.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.129+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b04050630 con 0x7f3b140c4a10 2023-10-30T12:51:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1414dae0 con 0x7f3b140c4a10 2023-10-30T12:51:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b0affd640 1 -- 172.21.15.8:0/345271169 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5736 (secure 0 0 0) 0x7f3b04018020 con 0x7f3b140c4a10 2023-10-30T12:51:21.324 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:21.324 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:18.315297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:21.325 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 35 2023-10-30T12:51:21.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 msgr2=0x7f3af8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 0x7f3af8068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b2000 tx=0x7f3b10008040 comp rx=0 tx=0).stop 2023-10-30T12:51:21.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 msgr2=0x7f3b14154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:21.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400c4e0 tx=0x7f3b0400c9a0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 shutdown_connections 2023-10-30T12:51:21.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3af80664a0 0x7f3af8068960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3b14153c80 0x7f3b14155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b140c4a10 0x7f3b14154ce0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 --2- 172.21.15.8:0/345271169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b140c42a0 0x7f3b14154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:21.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 >> 172.21.15.8:0/345271169 conn(0x7f3b140715c0 msgr2=0x7f3b14158450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:21.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 shutdown_connections 2023-10-30T12:51:21.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:21.321+0000 7f3b1c3d3640 1 -- 172.21.15.8:0/345271169 wait complete. 2023-10-30T12:51:21.346 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:51:23.347 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:23.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 Processor -- start 2023-10-30T12:51:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 -- start start 2023-10-30T12:51:23.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6acc150830 0x7f6acc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6acc0cb7d0 0x7f6acc0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0c2310 con 0x7f6acc0c3e40 2023-10-30T12:51:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6acc156710 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad2e46640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6acc0c4360 con 0x7f6acc150830 2023-10-30T12:51:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6acbfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6acc0cb7d0 0x7f6acc0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad13e3640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f6acc0c3e40 0x7f6acc070520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55254/0 (socket says 172.21.15.8:55254) 2023-10-30T12:51:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad13e3640 1 -- 172.21.15.8:0/1997079452 learned_addr learned my addr 172.21.15.8:0/1997079452 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.497+0000 7f6ad0be2640 1 --2- 172.21.15.8:0/1997079452 >> v2:172.21.15.46:3300/0 conn(0x7f6acc150830 0x7f6acc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acbfff640 1 -- 172.21.15.8:0/1997079452 >> v2:172.21.15.46:3300/0 conn(0x7f6acc150830 msgr2=0x7f6acc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acbfff640 1 --2- 172.21.15.8:0/1997079452 >> v2:172.21.15.46:3300/0 conn(0x7f6acc150830 0x7f6acc06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acbfff640 1 -- 172.21.15.8:0/1997079452 >> v1:172.21.15.8:6789/0 conn(0x7f6acc0c3e40 legacy=0x7f6acc070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acbfff640 1 -- 172.21.15.8:0/1997079452 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc1569b0 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acbfff640 1 --2- 172.21.15.8:0/1997079452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc0c3900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6acc06fa40 tx=0x7f6ab801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7f3a48239d1103 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acb7fe640 1 -- 172.21.15.8:0/1997079452 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80a8950 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acb7fe640 1 -- 172.21.15.8:0/1997079452 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab80a8ab0 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6acb7fe640 1 -- 172.21.15.8:0/1997079452 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80a8e40 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 -- 172.21.15.8:0/1997079452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 msgr2=0x7f6acc0c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/1997079452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc0c3900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6acc06fa40 tx=0x7f6ab801ec50 comp rx=0 tx=0).stop 2023-10-30T12:51:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 -- 172.21.15.8:0/1997079452 shutdown_connections 2023-10-30T12:51:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/1997079452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc0c3900 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/1997079452 >> v2:172.21.15.46:3300/0 conn(0x7f6acc150830 0x7f6acc06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 -- 172.21.15.8:0/1997079452 >> 172.21.15.8:0/1997079452 conn(0x7f6acc0cac70 msgr2=0x7f6acc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:23.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 -- 172.21.15.8:0/1997079452 shutdown_connections 2023-10-30T12:51:23.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.501+0000 7f6ad2e46640 1 -- 172.21.15.8:0/1997079452 wait complete. 2023-10-30T12:51:23.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 Processor -- start 2023-10-30T12:51:23.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 -- start start 2023-10-30T12:51:23.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6acc0c3e40 0x7f6acc1ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc150830 0x7f6acc146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1458a0 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1efde0 con 0x7f6acc150830 2023-10-30T12:51:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c8af0 con 0x7f6acc0c3e40 2023-10-30T12:51:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53436/0 (socket says 172.21.15.8:53436) 2023-10-30T12:51:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 -- 172.21.15.8:0/571938793 learned_addr learned my addr 172.21.15.8:0/571938793 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad13e3640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc150830 0x7f6acc146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad0be2640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6acc0c3e40 0x7f6acc1ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 -- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6acc0c3e40 msgr2=0x7f6acc1ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6acc0c3e40 0x7f6acc1ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 -- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc150830 msgr2=0x7f6acc146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc150830 0x7f6acc146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 -- 172.21.15.8:0/571938793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8009420 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6acbfff640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b3040 tx=0x7f6ab801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80b9630 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab80b97b0 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc0c8d90 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.505+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80b9b60 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc0c9350 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc066510 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6ab80b7040 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ac97fa640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 0x7f6ab0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f6ab81220e0 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ad0be2640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 0x7f6ab0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:23.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.509+0000 7f6ad0be2640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 0x7f6ab0068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000da10 tx=0x7f6ac0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:23.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.513+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6ab80ef350 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6acc04e170 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ac97fa640 1 -- 172.21.15.8:0/571938793 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5736 (secure 0 0 0) 0x7f6ab80f2970 con 0x7f6acc0cb7d0 2023-10-30T12:51:23.708 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:23.709 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:18.315297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:23.709 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 35 2023-10-30T12:51:23.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 msgr2=0x7f6ab0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 0x7f6ab0068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000da10 tx=0x7f6ac0008040 comp rx=0 tx=0).stop 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 msgr2=0x7f6acc14e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b3040 tx=0x7f6ab801b9f0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 shutdown_connections 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ab00664a0 0x7f6ab0068960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc150830 0x7f6acc146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6acc0cb7d0 0x7f6acc14e9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 --2- 172.21.15.8:0/571938793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6acc0c3e40 0x7f6acc1ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 >> 172.21.15.8:0/571938793 conn(0x7f6acc0cac70 msgr2=0x7f6acc0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 shutdown_connections 2023-10-30T12:51:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:23.705+0000 7f6ad2e46640 1 -- 172.21.15.8:0/571938793 wait complete. 2023-10-30T12:51:23.730 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 35, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 35, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4303": { "addr": "172.21.15.8:6837/2926864154", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 2926864154, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 2926864154, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4303, "incarnation": 4, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 33 }, "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:clientreplay", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:51:18.315297+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4303, "mds_1": 5079, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:51:25.731 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:25.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 Processor -- start 2023-10-30T12:51:25.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 -- start start 2023-10-30T12:51:25.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 0x7f8f4406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 0x7f8f440709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440c2310 con 0x7f8f440cb7d0 2023-10-30T12:51:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44156710 con 0x7f8f44150830 2023-10-30T12:51:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f4a69a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440c4360 con 0x7f8f44070520 2023-10-30T12:51:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f437fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8f440cb7d0 0x7f8f440c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55260/0 (socket says 172.21.15.8:55260) 2023-10-30T12:51:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f437fe640 1 -- 172.21.15.8:0/3878699114 learned_addr learned my addr 172.21.15.8:0/3878699114 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:51:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f48c37640 1 --2- 172.21.15.8:0/3878699114 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 0x7f8f440709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f43fff640 1 --2- 172.21.15.8:0/3878699114 >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 0x7f8f4406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330578463 0 0) 0x7f8f440c2310 con 0x7f8f440cb7d0 2023-10-30T12:51:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f2c003530 con 0x7f8f440cb7d0 2023-10-30T12:51:25.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f48c37640 1 -- 172.21.15.8:0/3878699114 >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 msgr2=0x7f8f4406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:25.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f48c37640 1 --2- 172.21.15.8:0/3878699114 >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 0x7f8f4406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1480619544 0 0) 0x7f8f2c003530 con 0x7f8f440cb7d0 2023-10-30T12:51:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f48c37640 1 -- 172.21.15.8:0/3878699114 >> v1:172.21.15.8:6789/0 conn(0x7f8f440cb7d0 legacy=0x7f8f440c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f48c37640 1 -- 172.21.15.8:0/3878699114 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f441569b0 con 0x7f8f44070520 2023-10-30T12:51:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.885+0000 7f8f43fff640 1 --2- 172.21.15.8:0/3878699114 >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 0x7f8f4406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:51:25.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f48c37640 1 --2- 172.21.15.8:0/3878699114 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 0x7f8f440709b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01f060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91caf91502551b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:25.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38001680 con 0x7f8f44070520 2023-10-30T12:51:25.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0a8bb0 con 0x7f8f44070520 2023-10-30T12:51:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f42ffd640 1 -- 172.21.15.8:0/3878699114 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0b0440 con 0x7f8f44070520 2023-10-30T12:51:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3878699114 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 msgr2=0x7f8f440709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3878699114 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 0x7f8f440709b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01f060 comp rx=0 tx=0).stop 2023-10-30T12:51:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3878699114 shutdown_connections 2023-10-30T12:51:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3878699114 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44070520 0x7f8f440709b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3878699114 >> v2:172.21.15.46:3300/0 conn(0x7f8f44150830 0x7f8f4406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3878699114 >> 172.21.15.8:0/3878699114 conn(0x7f8f440cac70 msgr2=0x7f8f440cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:25.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3878699114 shutdown_connections 2023-10-30T12:51:25.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.889+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3878699114 wait complete. 2023-10-30T12:51:25.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 Processor -- start 2023-10-30T12:51:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 -- start start 2023-10-30T12:51:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f440cb7d0 0x7f8f441e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44150830 0x7f8f44144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441e1bd0 con 0x7f8f44070520 2023-10-30T12:51:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44154450 con 0x7f8f440cb7d0 2023-10-30T12:51:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441e0150 con 0x7f8f44150830 2023-10-30T12:51:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53442/0 (socket says 172.21.15.8:53442) 2023-10-30T12:51:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 -- 172.21.15.8:0/3670347259 learned_addr learned my addr 172.21.15.8:0/3670347259 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f48c37640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44150830 0x7f8f44144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f437fe640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f440cb7d0 0x7f8f441e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 -- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44150830 msgr2=0x7f8f44144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44150830 0x7f8f44144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 -- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f440cb7d0 msgr2=0x7f8f441e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f440cb7d0 0x7f8f441e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 -- 172.21.15.8:0/3670347259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009420 con 0x7f8f44070520 2023-10-30T12:51:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f43fff640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e980 tx=0x7f8f3400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ca00 con 0x7f8f44070520 2023-10-30T12:51:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400cb60 con 0x7f8f44070520 2023-10-30T12:51:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34013470 con 0x7f8f44070520 2023-10-30T12:51:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.893+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c009a70 con 0x7f8f44070520 2023-10-30T12:51:25.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4414f330 con 0x7f8f44070520 2023-10-30T12:51:25.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4406c960 con 0x7f8f44070520 2023-10-30T12:51:25.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8f340135d0 con 0x7f8f44070520 2023-10-30T12:51:25.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f40ff9640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 0x7f8f30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:25.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f8f34084690 con 0x7f8f44070520 2023-10-30T12:51:25.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.897+0000 7f8f437fe640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 0x7f8f30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:25.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.901+0000 7f8f437fe640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 0x7f8f30068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f4414f9f0 tx=0x7f8f38001dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:25.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:25.901+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8f34051920 con 0x7f8f44070520 2023-10-30T12:51:26.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f4404e170 con 0x7f8f44070520 2023-10-30T12:51:26.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f40ff9640 1 -- 172.21.15.8:0/3670347259 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5730 (secure 0 0 0) 0x7f8f34054f40 con 0x7f8f44070520 2023-10-30T12:51:26.100 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:26.100 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:24.368240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2023-10-30T12:51:26.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 msgr2=0x7f8f30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:26.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 0x7f8f30068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f4414f9f0 tx=0x7f8f38001dd0 comp rx=0 tx=0).stop 2023-10-30T12:51:26.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 msgr2=0x7f8f4414b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:26.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e980 tx=0x7f8f3400ee40 comp rx=0 tx=0).stop 2023-10-30T12:51:26.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 shutdown_connections 2023-10-30T12:51:26.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8f300664a0 0x7f8f30068960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:26.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f440cb7d0 0x7f8f441e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:26.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8f44150830 0x7f8f44144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:26.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 --2- 172.21.15.8:0/3670347259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f44070520 0x7f8f4414b100 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:26.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 >> 172.21.15.8:0/3670347259 conn(0x7f8f440cac70 msgr2=0x7f8f441e1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:26.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 shutdown_connections 2023-10-30T12:51:26.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:26.097+0000 7f8f4a69a640 1 -- 172.21.15.8:0/3670347259 wait complete. 2023-10-30T12:51:26.121 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=4276 2023-10-30T12:51:26.121 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.f 2023-10-30T12:51:27.418 INFO:tasks.ceph.mds.a.smithi008.stderr:2023-10-30T12:51:27.413+0000 7fe0693b9640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:51:30.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/34' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/38' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/11' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/16' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/32' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/30' 2023-10-30T12:51:30.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/14' 2023-10-30T12:51:30.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/18' 2023-10-30T12:51:30.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/13' 2023-10-30T12:51:30.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/33' 2023-10-30T12:51:30.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/17' 2023-10-30T12:51:30.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/35' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './196/37' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './196' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/19' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/15' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/31' 2023-10-30T12:51:30.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/36' 2023-10-30T12:51:30.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/20' 2023-10-30T12:51:30.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/12' 2023-10-30T12:51:30.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/39' 2023-10-30T12:51:30.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/10' 2023-10-30T12:51:30.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/34' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/38' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/11' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/16' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/32' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/30' 2023-10-30T12:51:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/14' 2023-10-30T12:51:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/18' 2023-10-30T12:51:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/13' 2023-10-30T12:51:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/33' 2023-10-30T12:51:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/17' 2023-10-30T12:51:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/35' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './197/37' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './197' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/19' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/15' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/31' 2023-10-30T12:51:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/36' 2023-10-30T12:51:30.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/20' 2023-10-30T12:51:30.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/12' 2023-10-30T12:51:30.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/39' 2023-10-30T12:51:30.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/10' 2023-10-30T12:51:30.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/34' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/38' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/11' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/16' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/32' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/30' 2023-10-30T12:51:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/14' 2023-10-30T12:51:30.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/18' 2023-10-30T12:51:30.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/13' 2023-10-30T12:51:30.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/33' 2023-10-30T12:51:30.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/17' 2023-10-30T12:51:30.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/35' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './198/37' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './198' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/19' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/15' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/31' 2023-10-30T12:51:30.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/36' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/20' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/12' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/39' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/10' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/34' 2023-10-30T12:51:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/38' 2023-10-30T12:51:30.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/11' 2023-10-30T12:51:30.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/16' 2023-10-30T12:51:30.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/32' 2023-10-30T12:51:30.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/30' 2023-10-30T12:51:30.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/14' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/18' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/13' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/33' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/17' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/35' 2023-10-30T12:51:30.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './199/37' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './199' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/21' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/27' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/31' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/36' 2023-10-30T12:51:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/20' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/39' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/34' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/38' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/28' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/24' 2023-10-30T12:51:30.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/29' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/32' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/22' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/25' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/30' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/33' 2023-10-30T12:51:30.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/23' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/35' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/26' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './200/37' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './200' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/21' 2023-10-30T12:51:30.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/27' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/31' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/36' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/20' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/39' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/34' 2023-10-30T12:51:30.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/38' 2023-10-30T12:51:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/28' 2023-10-30T12:51:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/24' 2023-10-30T12:51:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/29' 2023-10-30T12:51:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/32' 2023-10-30T12:51:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/22' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/25' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/30' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/33' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/23' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/35' 2023-10-30T12:51:30.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/26' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './201/37' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './201' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/21' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/27' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/31' 2023-10-30T12:51:30.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/36' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/20' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/39' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/34' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/38' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/28' 2023-10-30T12:51:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/24' 2023-10-30T12:51:30.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/29' 2023-10-30T12:51:30.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/32' 2023-10-30T12:51:30.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/22' 2023-10-30T12:51:30.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/25' 2023-10-30T12:51:30.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/30' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/33' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/23' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/35' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/26' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './202/37' 2023-10-30T12:51:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './202' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/21' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/27' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/31' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/36' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/20' 2023-10-30T12:51:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/39' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/34' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/38' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/28' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/24' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/29' 2023-10-30T12:51:30.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/32' 2023-10-30T12:51:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/22' 2023-10-30T12:51:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/25' 2023-10-30T12:51:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/30' 2023-10-30T12:51:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/33' 2023-10-30T12:51:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/23' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/35' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/26' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './203/37' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './203' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/21' 2023-10-30T12:51:30.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/27' 2023-10-30T12:51:30.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/31' 2023-10-30T12:51:30.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/36' 2023-10-30T12:51:30.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/20' 2023-10-30T12:51:30.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/39' 2023-10-30T12:51:30.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/34' 2023-10-30T12:51:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/38' 2023-10-30T12:51:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/28' 2023-10-30T12:51:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/24' 2023-10-30T12:51:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/29' 2023-10-30T12:51:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/32' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/22' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/25' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/30' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/33' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/23' 2023-10-30T12:51:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/35' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/26' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './204/37' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './204' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/21' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/27' 2023-10-30T12:51:30.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/31' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/36' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/20' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/39' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/34' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/38' 2023-10-30T12:51:30.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/28' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/24' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/29' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/32' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/22' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/25' 2023-10-30T12:51:30.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/30' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/33' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/23' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/35' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/26' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './205/37' 2023-10-30T12:51:30.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './205' 2023-10-30T12:51:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/21' 2023-10-30T12:51:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/27' 2023-10-30T12:51:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/31' 2023-10-30T12:51:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/36' 2023-10-30T12:51:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/20' 2023-10-30T12:51:30.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/39' 2023-10-30T12:51:32.490 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/34' 2023-10-30T12:51:32.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/38' 2023-10-30T12:51:32.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/28' 2023-10-30T12:51:32.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/24' 2023-10-30T12:51:32.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/29' 2023-10-30T12:51:32.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/32' 2023-10-30T12:51:32.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/22' 2023-10-30T12:51:32.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/25' 2023-10-30T12:51:32.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/30' 2023-10-30T12:51:32.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/33' 2023-10-30T12:51:32.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/23' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/35' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/26' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './206/37' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './206' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/21' 2023-10-30T12:51:32.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/27' 2023-10-30T12:51:32.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/31' 2023-10-30T12:51:32.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/36' 2023-10-30T12:51:32.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/20' 2023-10-30T12:51:32.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/39' 2023-10-30T12:51:32.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/34' 2023-10-30T12:51:32.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/38' 2023-10-30T12:51:32.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/28' 2023-10-30T12:51:32.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/24' 2023-10-30T12:51:32.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/29' 2023-10-30T12:51:32.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/32' 2023-10-30T12:51:32.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/22' 2023-10-30T12:51:32.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/25' 2023-10-30T12:51:32.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/30' 2023-10-30T12:51:32.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/33' 2023-10-30T12:51:32.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/23' 2023-10-30T12:51:32.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/35' 2023-10-30T12:51:32.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/26' 2023-10-30T12:51:32.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './207/37' 2023-10-30T12:51:32.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './207' 2023-10-30T12:51:32.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/21' 2023-10-30T12:51:32.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/27' 2023-10-30T12:51:32.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/31' 2023-10-30T12:51:32.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/36' 2023-10-30T12:51:32.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/20' 2023-10-30T12:51:32.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/39' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/34' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/38' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/28' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/24' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/29' 2023-10-30T12:51:32.499 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/32' 2023-10-30T12:51:32.500 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/22' 2023-10-30T12:51:32.500 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/25' 2023-10-30T12:51:32.500 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/30' 2023-10-30T12:51:32.500 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/33' 2023-10-30T12:51:32.500 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/23' 2023-10-30T12:51:32.501 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/35' 2023-10-30T12:51:32.501 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/26' 2023-10-30T12:51:32.501 INFO:tasks.workunit.client.0.smithi008.stdout:removed './208/37' 2023-10-30T12:51:32.501 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './208' 2023-10-30T12:51:32.501 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/21' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/27' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/31' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/36' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/20' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/39' 2023-10-30T12:51:32.502 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/34' 2023-10-30T12:51:32.503 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/38' 2023-10-30T12:51:32.503 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/28' 2023-10-30T12:51:32.503 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/24' 2023-10-30T12:51:32.503 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/29' 2023-10-30T12:51:32.503 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/32' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/22' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/25' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/30' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/33' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/23' 2023-10-30T12:51:32.504 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/35' 2023-10-30T12:51:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/26' 2023-10-30T12:51:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:removed './209/37' 2023-10-30T12:51:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './209' 2023-10-30T12:51:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/21' 2023-10-30T12:51:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/27' 2023-10-30T12:51:32.506 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/31' 2023-10-30T12:51:32.506 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/36' 2023-10-30T12:51:32.506 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/20' 2023-10-30T12:51:32.506 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/39' 2023-10-30T12:51:32.506 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/34' 2023-10-30T12:51:32.507 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/38' 2023-10-30T12:51:32.507 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/28' 2023-10-30T12:51:32.507 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/24' 2023-10-30T12:51:32.507 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/29' 2023-10-30T12:51:32.507 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/32' 2023-10-30T12:51:32.508 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/22' 2023-10-30T12:51:32.508 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/25' 2023-10-30T12:51:32.508 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/30' 2023-10-30T12:51:32.508 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/33' 2023-10-30T12:51:32.508 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/23' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/35' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/26' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed './210/37' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './210' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/21' 2023-10-30T12:51:32.509 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/27' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/31' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/36' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/20' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/39' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/34' 2023-10-30T12:51:32.510 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/38' 2023-10-30T12:51:32.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/28' 2023-10-30T12:51:32.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/24' 2023-10-30T12:51:32.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/29' 2023-10-30T12:51:32.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/32' 2023-10-30T12:51:32.511 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/22' 2023-10-30T12:51:32.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/25' 2023-10-30T12:51:32.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/30' 2023-10-30T12:51:32.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/33' 2023-10-30T12:51:32.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/23' 2023-10-30T12:51:32.512 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/35' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/26' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './211/37' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './211' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/21' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/27' 2023-10-30T12:51:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/31' 2023-10-30T12:51:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/36' 2023-10-30T12:51:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/20' 2023-10-30T12:51:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/39' 2023-10-30T12:51:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/34' 2023-10-30T12:51:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/38' 2023-10-30T12:51:32.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/28' 2023-10-30T12:51:32.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/24' 2023-10-30T12:51:32.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/29' 2023-10-30T12:51:32.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/32' 2023-10-30T12:51:32.515 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/22' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/25' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/30' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/33' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/23' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/35' 2023-10-30T12:51:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/26' 2023-10-30T12:51:32.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './212/37' 2023-10-30T12:51:32.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './212' 2023-10-30T12:51:32.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/21' 2023-10-30T12:51:32.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/27' 2023-10-30T12:51:32.517 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/31' 2023-10-30T12:51:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/36' 2023-10-30T12:51:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/20' 2023-10-30T12:51:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/39' 2023-10-30T12:51:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/34' 2023-10-30T12:51:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/38' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/28' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/24' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/29' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/32' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/22' 2023-10-30T12:51:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/25' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/30' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/33' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/23' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/35' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/26' 2023-10-30T12:51:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:removed './213/37' 2023-10-30T12:51:32.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './213' 2023-10-30T12:51:32.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/21' 2023-10-30T12:51:32.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/27' 2023-10-30T12:51:32.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/31' 2023-10-30T12:51:32.521 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/36' 2023-10-30T12:51:32.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/20' 2023-10-30T12:51:32.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/39' 2023-10-30T12:51:32.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/34' 2023-10-30T12:51:32.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/38' 2023-10-30T12:51:32.522 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/28' 2023-10-30T12:51:32.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/24' 2023-10-30T12:51:32.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/29' 2023-10-30T12:51:32.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/32' 2023-10-30T12:51:32.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/22' 2023-10-30T12:51:32.523 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/25' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/30' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/33' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/23' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/35' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/26' 2023-10-30T12:51:32.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './214/37' 2023-10-30T12:51:32.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './214' 2023-10-30T12:51:32.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/21' 2023-10-30T12:51:32.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/27' 2023-10-30T12:51:32.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/31' 2023-10-30T12:51:32.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/36' 2023-10-30T12:51:32.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/20' 2023-10-30T12:51:32.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/39' 2023-10-30T12:51:32.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/34' 2023-10-30T12:51:32.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/38' 2023-10-30T12:51:32.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/28' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/24' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/29' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/32' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/22' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/25' 2023-10-30T12:51:32.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/30' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/33' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/23' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/35' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/26' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './215/37' 2023-10-30T12:51:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './215' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/21' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/27' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/31' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/36' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/20' 2023-10-30T12:51:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/39' 2023-10-30T12:51:32.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/34' 2023-10-30T12:51:32.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/38' 2023-10-30T12:51:34.122 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-10-30T12:51:34.123 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-30T12:51:34.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-30T12:51:34.126 INFO:tasks.ceph.mds.f:Started 2023-10-30T12:51:34.126 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:51:34.206 INFO:tasks.ceph.mds.f.smithi046.stdout:starting mds.f at 2023-10-30T12:51:34.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 Processor -- start 2023-10-30T12:51:34.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 -- start start 2023-10-30T12:51:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcaf81503b0 0x7fcaf80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcaf80c8970 0x7fcaf806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf815dc30 con 0x7fcaf8150790 2023-10-30T12:51:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf80c8e00 con 0x7fcaf81503b0 2023-10-30T12:51:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcafc89e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf80c4220 con 0x7fcaf80c8970 2023-10-30T12:51:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcaf659c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcaf81503b0 0x7fcaf80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcaf659c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcaf81503b0 0x7fcaf80c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42762/0 (socket says 172.21.15.8:42762) 2023-10-30T12:51:34.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcaf5d9b640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fcaf8150790 0x7fcaf806ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38110/0 (socket says 172.21.15.8:38110) 2023-10-30T12:51:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.277+0000 7fcaf659c640 1 -- 172.21.15.8:0/1449605849 learned_addr learned my addr 172.21.15.8:0/1449605849 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf6d9d640 1 --2- 172.21.15.8:0/1449605849 >> v2:172.21.15.46:3300/0 conn(0x7fcaf80c8970 0x7fcaf806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf659c640 1 -- 172.21.15.8:0/1449605849 >> v2:172.21.15.46:3300/0 conn(0x7fcaf80c8970 msgr2=0x7fcaf806bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:51:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf659c640 1 --2- 172.21.15.8:0/1449605849 >> v2:172.21.15.46:3300/0 conn(0x7fcaf80c8970 0x7fcaf806bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf659c640 1 -- 172.21.15.8:0/1449605849 >> v1:172.21.15.8:6789/0 conn(0x7fcaf8150790 legacy=0x7fcaf806ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf659c640 1 -- 172.21.15.8:0/1449605849 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf806e620 con 0x7fcaf81503b0 2023-10-30T12:51:34.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.281+0000 7fcaf559a640 1 -- 172.21.15.8:0/1449605849 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833411952 0 0) 0x7fcaf815dc30 con 0x7fcaf8150790 2023-10-30T12:51:34.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcaf659c640 1 --2- 172.21.15.8:0/1449605849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf80c7b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42c0f81a0af8c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcaf559a640 1 -- 172.21.15.8:0/1449605849 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80a89d0 con 0x7fcaf81503b0 2023-10-30T12:51:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcaf559a640 1 -- 172.21.15.8:0/1449605849 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae80a8b30 con 0x7fcaf81503b0 2023-10-30T12:51:34.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcaf559a640 1 -- 172.21.15.8:0/1449605849 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b0440 con 0x7fcaf81503b0 2023-10-30T12:51:34.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 -- 172.21.15.8:0/1449605849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 msgr2=0x7fcaf80c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 --2- 172.21.15.8:0/1449605849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf80c7b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae801ecd0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 -- 172.21.15.8:0/1449605849 shutdown_connections 2023-10-30T12:51:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 --2- 172.21.15.8:0/1449605849 >> v2:172.21.15.46:3300/0 conn(0x7fcaf80c8970 0x7fcaf806bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 --2- 172.21.15.8:0/1449605849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf80c7b30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 -- 172.21.15.8:0/1449605849 >> 172.21.15.8:0/1449605849 conn(0x7fcaf806fbe0 msgr2=0x7fcaf806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 -- 172.21.15.8:0/1449605849 shutdown_connections 2023-10-30T12:51:34.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.285+0000 7fcafc89e640 1 -- 172.21.15.8:0/1449605849 wait complete. 2023-10-30T12:51:34.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 Processor -- start 2023-10-30T12:51:34.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- start start 2023-10-30T12:51:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 0x7fcaf8159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcaf8150790 0x7fcaf8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81f0070 con 0x7fcaf81503b0 2023-10-30T12:51:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81ef3f0 con 0x7fcaf80c8970 2023-10-30T12:51:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81ef6d0 con 0x7fcaf8150790 2023-10-30T12:51:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42770/0 (socket says 172.21.15.8:42770) 2023-10-30T12:51:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 -- 172.21.15.8:0/119867710 learned_addr learned my addr 172.21.15.8:0/119867710 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:51:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf659c640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 0x7fcaf8159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf6d9d640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcaf8150790 0x7fcaf8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 -- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcaf8150790 msgr2=0x7fcaf8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcaf8150790 0x7fcaf8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 -- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 msgr2=0x7fcaf8159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 0x7fcaf8159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 -- 172.21.15.8:0/119867710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80094d0 con 0x7fcaf81503b0 2023-10-30T12:51:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf659c640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 0x7fcaf8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcaf5d9b640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0011f0 tx=0x7fcaec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0074e0 con 0x7fcaf81503b0 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec007640 con 0x7fcaf81503b0 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf81ef9b0 con 0x7fcaf81503b0 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec004a10 con 0x7fcaf81503b0 2023-10-30T12:51:34.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.289+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf815a5d0 con 0x7fcaf81503b0 2023-10-30T12:51:34.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf8157690 con 0x7fcaf81503b0 2023-10-30T12:51:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcaec004b70 con 0x7fcaf81503b0 2023-10-30T12:51:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcae6ffd640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 0x7fcad8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:51:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fcaec0831d0 con 0x7fcaf81503b0 2023-10-30T12:51:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcaf659c640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 0x7fcad8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:51:34.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.293+0000 7fcaf659c640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 0x7fcad8068990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b2000 tx=0x7fcae8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:51:34.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.297+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcaf8157690 con 0x7fcaf81503b0 2023-10-30T12:51:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.493+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf8157690 con 0x7fcaf81503b0 2023-10-30T12:51:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.493+0000 7fcae6ffd640 1 -- 172.21.15.8:0/119867710 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6521 (secure 0 0 0) 0x7fcaf8157690 con 0x7fcaf81503b0 2023-10-30T12:51:34.497 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:51:34.498 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:51:24.368240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:51:34.499 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 37 2023-10-30T12:51:34.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 msgr2=0x7fcad8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 0x7fcad8068990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b2000 tx=0x7fcae8008040 comp rx=0 tx=0).stop 2023-10-30T12:51:34.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 msgr2=0x7fcaf8152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:51:34.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0011f0 tx=0x7fcaec00c9b0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 shutdown_connections 2023-10-30T12:51:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcad80664d0 0x7fcad8068990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcaf8150790 0x7fcaf8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcaf81503b0 0x7fcaf8152650 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 --2- 172.21.15.8:0/119867710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf80c8970 0x7fcaf8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:51:34.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 >> 172.21.15.8:0/119867710 conn(0x7fcaf806fbe0 msgr2=0x7fcaf80c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:51:34.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 shutdown_connections 2023-10-30T12:51:34.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:51:34.497+0000 7fcafc89e640 1 -- 172.21.15.8:0/119867710 wait complete. 2023-10-30T12:51:34.518 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-10-30T12:51:34.518 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 73 secs before thrashing 2023-10-30T12:51:36.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/28' 2023-10-30T12:51:36.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/24' 2023-10-30T12:51:36.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/29' 2023-10-30T12:51:36.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/32' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/22' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/25' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/30' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/33' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/23' 2023-10-30T12:51:36.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/35' 2023-10-30T12:51:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/26' 2023-10-30T12:51:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './216/37' 2023-10-30T12:51:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './216' 2023-10-30T12:51:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/21' 2023-10-30T12:51:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/27' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/31' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/36' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/20' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/39' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/34' 2023-10-30T12:51:36.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/38' 2023-10-30T12:51:36.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/28' 2023-10-30T12:51:36.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/24' 2023-10-30T12:51:36.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/29' 2023-10-30T12:51:36.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/32' 2023-10-30T12:51:36.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/22' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/25' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/30' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/33' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/23' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/35' 2023-10-30T12:51:36.954 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/26' 2023-10-30T12:51:36.955 INFO:tasks.workunit.client.0.smithi008.stdout:removed './217/37' 2023-10-30T12:51:36.955 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './217' 2023-10-30T12:51:36.955 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/21' 2023-10-30T12:51:36.955 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/27' 2023-10-30T12:51:36.955 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/31' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/36' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/20' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/39' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/34' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/38' 2023-10-30T12:51:36.956 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/28' 2023-10-30T12:51:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/24' 2023-10-30T12:51:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/29' 2023-10-30T12:51:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/32' 2023-10-30T12:51:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/22' 2023-10-30T12:51:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/25' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/30' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/33' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/23' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/35' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/26' 2023-10-30T12:51:36.958 INFO:tasks.workunit.client.0.smithi008.stdout:removed './218/37' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './218' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/21' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/27' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/31' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/36' 2023-10-30T12:51:36.959 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/20' 2023-10-30T12:51:36.960 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/39' 2023-10-30T12:51:36.960 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/34' 2023-10-30T12:51:36.960 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/38' 2023-10-30T12:51:36.960 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/28' 2023-10-30T12:51:36.960 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/24' 2023-10-30T12:51:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/29' 2023-10-30T12:51:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/32' 2023-10-30T12:51:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/22' 2023-10-30T12:51:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/25' 2023-10-30T12:51:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/30' 2023-10-30T12:51:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/33' 2023-10-30T12:51:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/23' 2023-10-30T12:51:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/35' 2023-10-30T12:51:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/26' 2023-10-30T12:51:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:removed './219/37' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './219' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/21' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/27' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/31' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/36' 2023-10-30T12:51:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/20' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/39' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/34' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/38' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/28' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/24' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/29' 2023-10-30T12:51:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/32' 2023-10-30T12:51:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/22' 2023-10-30T12:51:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/25' 2023-10-30T12:51:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/30' 2023-10-30T12:51:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/33' 2023-10-30T12:51:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/23' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/35' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/26' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed './220/37' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './220' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/21' 2023-10-30T12:51:36.966 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/27' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/31' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/36' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/20' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/39' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/34' 2023-10-30T12:51:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/38' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/28' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/24' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/29' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/32' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/22' 2023-10-30T12:51:36.968 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/25' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/30' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/33' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/23' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/35' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/26' 2023-10-30T12:51:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:removed './221/37' 2023-10-30T12:51:36.970 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './221' 2023-10-30T12:51:36.970 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/21' 2023-10-30T12:51:36.970 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/27' 2023-10-30T12:51:36.970 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/31' 2023-10-30T12:51:36.970 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/36' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/20' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/39' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/34' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/38' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/28' 2023-10-30T12:51:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/24' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/29' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/32' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/22' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/25' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/30' 2023-10-30T12:51:36.972 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/33' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/23' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/35' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/26' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed './222/37' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './222' 2023-10-30T12:51:36.973 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/21' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/27' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/31' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/36' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/20' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/39' 2023-10-30T12:51:36.974 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/34' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/38' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/28' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/24' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/29' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/32' 2023-10-30T12:51:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/22' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/25' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/30' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/33' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/23' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/35' 2023-10-30T12:51:36.976 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/26' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './223/37' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './223' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/21' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/27' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/31' 2023-10-30T12:51:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/36' 2023-10-30T12:51:36.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/20' 2023-10-30T12:51:36.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/39' 2023-10-30T12:51:36.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/34' 2023-10-30T12:51:36.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/38' 2023-10-30T12:51:36.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/28' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/24' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/29' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/32' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/22' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/25' 2023-10-30T12:51:36.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/30' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/33' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/23' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/35' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/26' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './224/37' 2023-10-30T12:51:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './224' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/21' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/27' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/31' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/36' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/20' 2023-10-30T12:51:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/39' 2023-10-30T12:51:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/34' 2023-10-30T12:51:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/38' 2023-10-30T12:51:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/28' 2023-10-30T12:51:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/24' 2023-10-30T12:51:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/29' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/32' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/22' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/25' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/30' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/33' 2023-10-30T12:51:36.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/23' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/35' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/26' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './225/37' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './225' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/21' 2023-10-30T12:51:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/27' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/31' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/36' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/20' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/39' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/34' 2023-10-30T12:51:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/38' 2023-10-30T12:51:36.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/28' 2023-10-30T12:51:36.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/24' 2023-10-30T12:51:37.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/29' 2023-10-30T12:51:37.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/32' 2023-10-30T12:51:37.524 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/22' 2023-10-30T12:51:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/25' 2023-10-30T12:51:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/30' 2023-10-30T12:51:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/33' 2023-10-30T12:51:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/23' 2023-10-30T12:51:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/35' 2023-10-30T12:51:37.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/26' 2023-10-30T12:51:37.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './226/37' 2023-10-30T12:51:37.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './226' 2023-10-30T12:51:37.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/21' 2023-10-30T12:51:37.526 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/27' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/31' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/36' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/20' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/39' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/34' 2023-10-30T12:51:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/38' 2023-10-30T12:51:37.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/28' 2023-10-30T12:51:37.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/24' 2023-10-30T12:51:37.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/29' 2023-10-30T12:51:37.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/32' 2023-10-30T12:51:37.528 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/22' 2023-10-30T12:51:37.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/25' 2023-10-30T12:51:37.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/30' 2023-10-30T12:51:37.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/33' 2023-10-30T12:51:37.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/23' 2023-10-30T12:51:37.529 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/35' 2023-10-30T12:51:37.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/26' 2023-10-30T12:51:37.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './227/37' 2023-10-30T12:51:37.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './227' 2023-10-30T12:51:37.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/21' 2023-10-30T12:51:37.530 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/27' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/31' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/36' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/20' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/39' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/34' 2023-10-30T12:51:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/38' 2023-10-30T12:51:37.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/28' 2023-10-30T12:51:37.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/24' 2023-10-30T12:51:37.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/29' 2023-10-30T12:51:37.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/32' 2023-10-30T12:51:37.532 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/22' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/25' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/30' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/33' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/23' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/35' 2023-10-30T12:51:37.533 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/26' 2023-10-30T12:51:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './228/37' 2023-10-30T12:51:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './228' 2023-10-30T12:51:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/21' 2023-10-30T12:51:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/27' 2023-10-30T12:51:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/31' 2023-10-30T12:51:37.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/36' 2023-10-30T12:51:37.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/20' 2023-10-30T12:51:37.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/39' 2023-10-30T12:51:37.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/34' 2023-10-30T12:51:37.535 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/38' 2023-10-30T12:51:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/28' 2023-10-30T12:51:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/24' 2023-10-30T12:51:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/29' 2023-10-30T12:51:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/32' 2023-10-30T12:51:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/22' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/25' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/30' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/33' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/23' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/35' 2023-10-30T12:51:37.537 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/26' 2023-10-30T12:51:37.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './229/37' 2023-10-30T12:51:37.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './229' 2023-10-30T12:51:37.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/21' 2023-10-30T12:51:37.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/27' 2023-10-30T12:51:37.538 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/31' 2023-10-30T12:51:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/36' 2023-10-30T12:51:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/20' 2023-10-30T12:51:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/39' 2023-10-30T12:51:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/34' 2023-10-30T12:51:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/38' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/28' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/24' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/29' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/32' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/22' 2023-10-30T12:51:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/25' 2023-10-30T12:51:37.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/30' 2023-10-30T12:51:37.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/33' 2023-10-30T12:51:37.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/23' 2023-10-30T12:51:37.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/35' 2023-10-30T12:51:37.541 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/26' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './230/37' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './230' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/21' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/27' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/31' 2023-10-30T12:51:37.542 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/36' 2023-10-30T12:51:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/20' 2023-10-30T12:51:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/39' 2023-10-30T12:51:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/34' 2023-10-30T12:51:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/38' 2023-10-30T12:51:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/28' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/24' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/29' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/32' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/22' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/25' 2023-10-30T12:51:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/30' 2023-10-30T12:51:37.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/33' 2023-10-30T12:51:37.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/23' 2023-10-30T12:51:37.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/35' 2023-10-30T12:51:37.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/26' 2023-10-30T12:51:37.545 INFO:tasks.workunit.client.0.smithi008.stdout:removed './231/37' 2023-10-30T12:51:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './231' 2023-10-30T12:51:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/21' 2023-10-30T12:51:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/27' 2023-10-30T12:51:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/31' 2023-10-30T12:51:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/36' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/20' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/39' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/34' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/38' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/28' 2023-10-30T12:51:37.547 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/24' 2023-10-30T12:51:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/29' 2023-10-30T12:51:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/32' 2023-10-30T12:51:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/22' 2023-10-30T12:51:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/25' 2023-10-30T12:51:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/30' 2023-10-30T12:51:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/33' 2023-10-30T12:51:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/23' 2023-10-30T12:51:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/35' 2023-10-30T12:51:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/26' 2023-10-30T12:51:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:removed './232/37' 2023-10-30T12:51:37.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './232' 2023-10-30T12:51:37.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/21' 2023-10-30T12:51:37.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/27' 2023-10-30T12:51:37.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/31' 2023-10-30T12:51:37.550 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/36' 2023-10-30T12:51:37.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/20' 2023-10-30T12:51:37.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/39' 2023-10-30T12:51:37.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/34' 2023-10-30T12:51:37.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/38' 2023-10-30T12:51:37.551 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/28' 2023-10-30T12:51:37.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/24' 2023-10-30T12:51:37.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/29' 2023-10-30T12:51:37.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/32' 2023-10-30T12:51:37.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/22' 2023-10-30T12:51:37.552 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/25' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/30' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/33' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/23' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/35' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/26' 2023-10-30T12:51:37.553 INFO:tasks.workunit.client.0.smithi008.stdout:removed './233/37' 2023-10-30T12:51:37.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './233' 2023-10-30T12:51:37.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/21' 2023-10-30T12:51:37.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/27' 2023-10-30T12:51:37.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/31' 2023-10-30T12:51:37.554 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/36' 2023-10-30T12:51:37.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/20' 2023-10-30T12:51:37.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/39' 2023-10-30T12:51:37.557 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/34' 2023-10-30T12:51:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/38' 2023-10-30T12:51:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/28' 2023-10-30T12:51:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/24' 2023-10-30T12:51:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/29' 2023-10-30T12:51:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/32' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/22' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/25' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/30' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/33' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/23' 2023-10-30T12:51:37.559 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/35' 2023-10-30T12:51:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/26' 2023-10-30T12:51:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './234/37' 2023-10-30T12:51:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './234' 2023-10-30T12:51:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/21' 2023-10-30T12:51:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/27' 2023-10-30T12:51:37.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/31' 2023-10-30T12:51:37.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/36' 2023-10-30T12:51:37.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/20' 2023-10-30T12:51:37.561 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/39' 2023-10-30T12:51:37.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/34' 2023-10-30T12:51:37.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/38' 2023-10-30T12:51:37.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/28' 2023-10-30T12:51:37.562 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/24' 2023-10-30T12:51:37.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/29' 2023-10-30T12:51:37.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/32' 2023-10-30T12:51:37.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/22' 2023-10-30T12:51:37.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/25' 2023-10-30T12:51:37.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/30' 2023-10-30T12:51:37.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/33' 2023-10-30T12:51:37.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/23' 2023-10-30T12:51:37.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/35' 2023-10-30T12:51:37.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/26' 2023-10-30T12:51:37.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './235/37' 2023-10-30T12:51:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './235' 2023-10-30T12:51:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/21' 2023-10-30T12:51:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/27' 2023-10-30T12:51:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/31' 2023-10-30T12:51:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/36' 2023-10-30T12:51:37.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/20' 2023-10-30T12:51:37.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/39' 2023-10-30T12:51:37.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/34' 2023-10-30T12:51:37.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/38' 2023-10-30T12:51:37.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/28' 2023-10-30T12:51:37.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/24' 2023-10-30T12:51:37.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/29' 2023-10-30T12:51:37.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/32' 2023-10-30T12:51:40.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/22' 2023-10-30T12:51:40.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/25' 2023-10-30T12:51:40.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/30' 2023-10-30T12:51:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/33' 2023-10-30T12:51:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/23' 2023-10-30T12:51:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/35' 2023-10-30T12:51:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/26' 2023-10-30T12:51:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './236/37' 2023-10-30T12:51:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './236' 2023-10-30T12:51:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/21' 2023-10-30T12:51:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/27' 2023-10-30T12:51:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/31' 2023-10-30T12:51:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/36' 2023-10-30T12:51:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/20' 2023-10-30T12:51:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/39' 2023-10-30T12:51:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/34' 2023-10-30T12:51:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/38' 2023-10-30T12:51:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/28' 2023-10-30T12:51:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/24' 2023-10-30T12:51:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/29' 2023-10-30T12:51:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/32' 2023-10-30T12:51:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/22' 2023-10-30T12:51:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/25' 2023-10-30T12:51:40.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/30' 2023-10-30T12:51:40.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/33' 2023-10-30T12:51:40.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/23' 2023-10-30T12:51:40.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/35' 2023-10-30T12:51:40.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/26' 2023-10-30T12:51:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './237/37' 2023-10-30T12:51:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './237' 2023-10-30T12:51:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/21' 2023-10-30T12:51:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/27' 2023-10-30T12:51:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/31' 2023-10-30T12:51:40.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/36' 2023-10-30T12:51:40.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/20' 2023-10-30T12:51:40.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/39' 2023-10-30T12:51:40.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/34' 2023-10-30T12:51:40.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/38' 2023-10-30T12:51:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/28' 2023-10-30T12:51:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/24' 2023-10-30T12:51:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/29' 2023-10-30T12:51:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/32' 2023-10-30T12:51:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/22' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/25' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/30' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/33' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/23' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/35' 2023-10-30T12:51:40.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/26' 2023-10-30T12:51:40.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './238/37' 2023-10-30T12:51:40.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './238' 2023-10-30T12:51:40.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/21' 2023-10-30T12:51:40.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/27' 2023-10-30T12:51:40.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/31' 2023-10-30T12:51:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/36' 2023-10-30T12:51:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/20' 2023-10-30T12:51:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/39' 2023-10-30T12:51:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/34' 2023-10-30T12:51:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/38' 2023-10-30T12:51:40.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/28' 2023-10-30T12:51:40.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/24' 2023-10-30T12:51:40.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/29' 2023-10-30T12:51:40.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/32' 2023-10-30T12:51:40.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/22' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/25' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/30' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/33' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/23' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/35' 2023-10-30T12:51:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/26' 2023-10-30T12:51:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:removed './239/37' 2023-10-30T12:51:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './239' 2023-10-30T12:51:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/21' 2023-10-30T12:51:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/27' 2023-10-30T12:51:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/31' 2023-10-30T12:51:40.029 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/36' 2023-10-30T12:51:40.029 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/20' 2023-10-30T12:51:40.029 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/39' 2023-10-30T12:51:40.029 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/34' 2023-10-30T12:51:40.029 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/38' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/28' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/24' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/29' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/32' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/22' 2023-10-30T12:51:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/25' 2023-10-30T12:51:40.031 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/30' 2023-10-30T12:51:40.031 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/33' 2023-10-30T12:51:40.031 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/23' 2023-10-30T12:51:40.031 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/35' 2023-10-30T12:51:40.031 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/26' 2023-10-30T12:51:40.032 INFO:tasks.workunit.client.0.smithi008.stdout:removed './240/37' 2023-10-30T12:51:40.032 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './240' 2023-10-30T12:51:40.032 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/21' 2023-10-30T12:51:40.032 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/27' 2023-10-30T12:51:40.032 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/31' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/36' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/20' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/39' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/34' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/38' 2023-10-30T12:51:40.033 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/28' 2023-10-30T12:51:40.034 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/24' 2023-10-30T12:51:40.034 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/29' 2023-10-30T12:51:40.034 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/32' 2023-10-30T12:51:40.034 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/22' 2023-10-30T12:51:40.034 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/25' 2023-10-30T12:51:40.035 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/30' 2023-10-30T12:51:40.035 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/33' 2023-10-30T12:51:40.035 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/23' 2023-10-30T12:51:40.035 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/35' 2023-10-30T12:51:40.035 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/26' 2023-10-30T12:51:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:removed './241/37' 2023-10-30T12:51:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './241' 2023-10-30T12:51:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/21' 2023-10-30T12:51:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/27' 2023-10-30T12:51:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/31' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/36' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/20' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/39' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/34' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/38' 2023-10-30T12:51:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/28' 2023-10-30T12:51:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/24' 2023-10-30T12:51:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/29' 2023-10-30T12:51:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/32' 2023-10-30T12:51:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/22' 2023-10-30T12:51:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/25' 2023-10-30T12:51:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/30' 2023-10-30T12:51:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/33' 2023-10-30T12:51:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/23' 2023-10-30T12:51:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/35' 2023-10-30T12:51:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/26' 2023-10-30T12:51:40.040 INFO:tasks.workunit.client.0.smithi008.stdout:removed './242/37' 2023-10-30T12:51:40.040 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './242' 2023-10-30T12:51:40.040 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/21' 2023-10-30T12:51:40.040 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/27' 2023-10-30T12:51:40.040 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/31' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/36' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/20' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/39' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/34' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/38' 2023-10-30T12:51:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/28' 2023-10-30T12:51:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/24' 2023-10-30T12:51:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/29' 2023-10-30T12:51:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/32' 2023-10-30T12:51:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/22' 2023-10-30T12:51:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/25' 2023-10-30T12:51:40.043 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/30' 2023-10-30T12:51:40.043 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/33' 2023-10-30T12:51:40.043 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/23' 2023-10-30T12:51:40.043 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/35' 2023-10-30T12:51:40.043 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/26' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed './243/37' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './243' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/21' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/27' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/31' 2023-10-30T12:51:40.044 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/36' 2023-10-30T12:51:40.045 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/20' 2023-10-30T12:51:40.045 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/39' 2023-10-30T12:51:40.045 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/34' 2023-10-30T12:51:40.045 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/38' 2023-10-30T12:51:40.045 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/28' 2023-10-30T12:51:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/24' 2023-10-30T12:51:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/29' 2023-10-30T12:51:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/32' 2023-10-30T12:51:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/22' 2023-10-30T12:51:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/25' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/30' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/33' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/23' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/35' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/26' 2023-10-30T12:51:40.047 INFO:tasks.workunit.client.0.smithi008.stdout:removed './244/37' 2023-10-30T12:51:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './244' 2023-10-30T12:51:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/21' 2023-10-30T12:51:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/27' 2023-10-30T12:51:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/31' 2023-10-30T12:51:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/36' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/20' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/39' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/34' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/38' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/28' 2023-10-30T12:51:40.049 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/24' 2023-10-30T12:51:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/29' 2023-10-30T12:51:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/32' 2023-10-30T12:51:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/22' 2023-10-30T12:51:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/25' 2023-10-30T12:51:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/30' 2023-10-30T12:51:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/33' 2023-10-30T12:51:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/23' 2023-10-30T12:51:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/35' 2023-10-30T12:51:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/26' 2023-10-30T12:51:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:removed './245/37' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './245' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/21' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/27' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/31' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/36' 2023-10-30T12:51:40.052 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/20' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/39' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/34' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/38' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/28' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/24' 2023-10-30T12:51:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/29' 2023-10-30T12:51:40.054 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/32' 2023-10-30T12:51:40.054 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/22' 2023-10-30T12:51:42.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/25' 2023-10-30T12:51:42.757 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/30' 2023-10-30T12:51:42.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/33' 2023-10-30T12:51:42.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/23' 2023-10-30T12:51:42.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/35' 2023-10-30T12:51:42.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/26' 2023-10-30T12:51:42.758 INFO:tasks.workunit.client.0.smithi008.stdout:removed './246/37' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './246' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/21' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/27' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/31' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/36' 2023-10-30T12:51:42.759 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/20' 2023-10-30T12:51:42.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/39' 2023-10-30T12:51:42.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/34' 2023-10-30T12:51:42.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/38' 2023-10-30T12:51:42.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/28' 2023-10-30T12:51:42.760 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/24' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/29' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/32' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/22' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/25' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/30' 2023-10-30T12:51:42.761 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/33' 2023-10-30T12:51:42.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/23' 2023-10-30T12:51:42.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/35' 2023-10-30T12:51:42.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/26' 2023-10-30T12:51:42.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed './247/37' 2023-10-30T12:51:42.762 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './247' 2023-10-30T12:51:42.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/21' 2023-10-30T12:51:42.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/27' 2023-10-30T12:51:42.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/31' 2023-10-30T12:51:42.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/36' 2023-10-30T12:51:42.763 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/20' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/39' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/34' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/38' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/28' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/24' 2023-10-30T12:51:42.764 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/29' 2023-10-30T12:51:42.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/32' 2023-10-30T12:51:42.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/22' 2023-10-30T12:51:42.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/25' 2023-10-30T12:51:42.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/30' 2023-10-30T12:51:42.765 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/33' 2023-10-30T12:51:42.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/23' 2023-10-30T12:51:42.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/35' 2023-10-30T12:51:42.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/26' 2023-10-30T12:51:42.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed './248/37' 2023-10-30T12:51:42.766 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './248' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/21' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/27' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/31' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/36' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/20' 2023-10-30T12:51:42.767 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/39' 2023-10-30T12:51:42.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/34' 2023-10-30T12:51:42.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/38' 2023-10-30T12:51:42.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/28' 2023-10-30T12:51:42.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/24' 2023-10-30T12:51:42.768 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/29' 2023-10-30T12:51:42.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/32' 2023-10-30T12:51:42.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/22' 2023-10-30T12:51:42.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/25' 2023-10-30T12:51:42.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/30' 2023-10-30T12:51:42.769 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/33' 2023-10-30T12:51:42.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/23' 2023-10-30T12:51:42.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/35' 2023-10-30T12:51:42.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/26' 2023-10-30T12:51:42.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed './249/37' 2023-10-30T12:51:42.770 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './249' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/21' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/27' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/31' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/36' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/20' 2023-10-30T12:51:42.771 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/39' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/34' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/38' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/28' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/24' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/29' 2023-10-30T12:51:42.772 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/32' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/22' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/25' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/30' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/33' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/23' 2023-10-30T12:51:42.773 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/35' 2023-10-30T12:51:42.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/26' 2023-10-30T12:51:42.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './250/37' 2023-10-30T12:51:42.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './250' 2023-10-30T12:51:42.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/21' 2023-10-30T12:51:42.774 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/27' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/31' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/36' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/20' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/39' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/34' 2023-10-30T12:51:42.775 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/38' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/28' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/24' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/29' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/32' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/22' 2023-10-30T12:51:42.776 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/25' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/30' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/33' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/23' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/35' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/26' 2023-10-30T12:51:42.777 INFO:tasks.workunit.client.0.smithi008.stdout:removed './251/37' 2023-10-30T12:51:42.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './251' 2023-10-30T12:51:42.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/21' 2023-10-30T12:51:42.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/27' 2023-10-30T12:51:42.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/31' 2023-10-30T12:51:42.778 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/36' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/20' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/39' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/34' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/38' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/28' 2023-10-30T12:51:42.779 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/24' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/29' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/32' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/22' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/25' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/30' 2023-10-30T12:51:42.780 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/33' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/23' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/35' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/26' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './252/37' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './252' 2023-10-30T12:51:42.781 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/21' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/27' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/31' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/36' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/20' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/39' 2023-10-30T12:51:42.782 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/34' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/38' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/28' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/24' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/29' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/32' 2023-10-30T12:51:42.783 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/22' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/25' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/30' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/33' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/23' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/35' 2023-10-30T12:51:42.784 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/26' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './253/37' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './253' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/21' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/27' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/31' 2023-10-30T12:51:42.785 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/36' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/20' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/39' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/34' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/38' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/28' 2023-10-30T12:51:42.786 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/24' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/29' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/32' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/22' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/25' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/30' 2023-10-30T12:51:42.787 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/33' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/23' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/35' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/26' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './254/37' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './254' 2023-10-30T12:51:42.788 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/21' 2023-10-30T12:51:42.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/27' 2023-10-30T12:51:42.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/31' 2023-10-30T12:51:42.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/36' 2023-10-30T12:51:42.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/20' 2023-10-30T12:51:42.789 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/39' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/34' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/38' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/28' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/24' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/29' 2023-10-30T12:51:42.790 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/32' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/22' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/25' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/30' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/33' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/23' 2023-10-30T12:51:42.791 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/35' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/26' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './255/37' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './255' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/21' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/27' 2023-10-30T12:51:42.792 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/31' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/36' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/20' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/39' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/34' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/38' 2023-10-30T12:51:42.793 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/28' 2023-10-30T12:51:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/24' 2023-10-30T12:51:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/29' 2023-10-30T12:51:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/32' 2023-10-30T12:51:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/22' 2023-10-30T12:51:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/25' 2023-10-30T12:51:42.795 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/30' 2023-10-30T12:51:45.160 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:45.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:51:47.375 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/33' 2023-10-30T12:51:47.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/23' 2023-10-30T12:51:47.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/35' 2023-10-30T12:51:47.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/26' 2023-10-30T12:51:47.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed './256/37' 2023-10-30T12:51:47.376 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './256' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/21' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/27' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/31' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/36' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/20' 2023-10-30T12:51:47.377 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/39' 2023-10-30T12:51:47.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/34' 2023-10-30T12:51:47.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/38' 2023-10-30T12:51:47.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/28' 2023-10-30T12:51:47.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/24' 2023-10-30T12:51:47.378 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/29' 2023-10-30T12:51:47.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/32' 2023-10-30T12:51:47.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/22' 2023-10-30T12:51:47.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/25' 2023-10-30T12:51:47.379 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/30' 2023-10-30T12:51:47.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/33' 2023-10-30T12:51:47.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/23' 2023-10-30T12:51:47.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/35' 2023-10-30T12:51:47.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/26' 2023-10-30T12:51:47.380 INFO:tasks.workunit.client.0.smithi008.stdout:removed './257/37' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './257' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/21' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/27' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/31' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/36' 2023-10-30T12:51:47.381 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/20' 2023-10-30T12:51:47.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/39' 2023-10-30T12:51:47.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/34' 2023-10-30T12:51:47.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/38' 2023-10-30T12:51:47.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/28' 2023-10-30T12:51:47.382 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/24' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/29' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/32' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/22' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/25' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/30' 2023-10-30T12:51:47.383 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/33' 2023-10-30T12:51:47.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/23' 2023-10-30T12:51:47.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/35' 2023-10-30T12:51:47.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/26' 2023-10-30T12:51:47.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed './258/37' 2023-10-30T12:51:47.384 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './258' 2023-10-30T12:51:47.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/21' 2023-10-30T12:51:47.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/27' 2023-10-30T12:51:47.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/31' 2023-10-30T12:51:47.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/36' 2023-10-30T12:51:47.385 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/20' 2023-10-30T12:51:47.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/39' 2023-10-30T12:51:47.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/34' 2023-10-30T12:51:47.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/38' 2023-10-30T12:51:47.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/28' 2023-10-30T12:51:47.386 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/24' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/29' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/32' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/22' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/25' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/30' 2023-10-30T12:51:47.387 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/33' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/23' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/35' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/26' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './259/37' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './259' 2023-10-30T12:51:47.388 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/21' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/27' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/31' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/36' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/20' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/39' 2023-10-30T12:51:47.389 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/34' 2023-10-30T12:51:47.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/38' 2023-10-30T12:51:47.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/28' 2023-10-30T12:51:47.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/24' 2023-10-30T12:51:47.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/29' 2023-10-30T12:51:47.390 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/32' 2023-10-30T12:51:47.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/22' 2023-10-30T12:51:47.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/25' 2023-10-30T12:51:47.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/30' 2023-10-30T12:51:47.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/33' 2023-10-30T12:51:47.391 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/23' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/35' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/26' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './260/37' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './260' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/21' 2023-10-30T12:51:47.392 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/27' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/31' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/36' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/20' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/39' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/34' 2023-10-30T12:51:47.393 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/38' 2023-10-30T12:51:47.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/28' 2023-10-30T12:51:47.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/24' 2023-10-30T12:51:47.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/29' 2023-10-30T12:51:47.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/32' 2023-10-30T12:51:47.394 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/22' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/25' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/30' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/33' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/23' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/35' 2023-10-30T12:51:47.395 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/26' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './261/37' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './261' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/21' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/27' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/31' 2023-10-30T12:51:47.396 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/36' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/20' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/39' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/34' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/38' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/28' 2023-10-30T12:51:47.397 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/24' 2023-10-30T12:51:47.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/29' 2023-10-30T12:51:47.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/32' 2023-10-30T12:51:47.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/22' 2023-10-30T12:51:47.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/25' 2023-10-30T12:51:47.398 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/30' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/33' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/23' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/35' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/26' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed './262/37' 2023-10-30T12:51:47.399 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './262' 2023-10-30T12:51:47.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/21' 2023-10-30T12:51:47.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/27' 2023-10-30T12:51:47.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/31' 2023-10-30T12:51:47.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/36' 2023-10-30T12:51:47.400 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/20' 2023-10-30T12:51:47.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/39' 2023-10-30T12:51:47.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/34' 2023-10-30T12:51:47.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/38' 2023-10-30T12:51:47.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/28' 2023-10-30T12:51:47.401 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/24' 2023-10-30T12:51:47.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/29' 2023-10-30T12:51:47.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/32' 2023-10-30T12:51:47.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/22' 2023-10-30T12:51:47.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/25' 2023-10-30T12:51:47.402 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/30' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/33' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/23' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/35' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/26' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed './263/37' 2023-10-30T12:51:47.403 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './263' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/21' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/27' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/31' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/36' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/20' 2023-10-30T12:51:47.404 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/39' 2023-10-30T12:51:47.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/34' 2023-10-30T12:51:47.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/38' 2023-10-30T12:51:47.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/28' 2023-10-30T12:51:47.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/24' 2023-10-30T12:51:47.405 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/29' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/32' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/22' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/25' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/30' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/33' 2023-10-30T12:51:47.406 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/23' 2023-10-30T12:51:47.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/35' 2023-10-30T12:51:47.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/26' 2023-10-30T12:51:47.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './264/37' 2023-10-30T12:51:47.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './264' 2023-10-30T12:51:47.407 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/21' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/27' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/31' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/36' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/20' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/39' 2023-10-30T12:51:47.408 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/34' 2023-10-30T12:51:47.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/38' 2023-10-30T12:51:47.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/28' 2023-10-30T12:51:47.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/24' 2023-10-30T12:51:47.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/29' 2023-10-30T12:51:47.409 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/32' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/22' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/25' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/30' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/33' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/23' 2023-10-30T12:51:47.410 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/35' 2023-10-30T12:51:47.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/26' 2023-10-30T12:51:47.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './265/37' 2023-10-30T12:51:47.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './265' 2023-10-30T12:51:47.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/21' 2023-10-30T12:51:47.411 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/27' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/31' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/36' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/20' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/39' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/34' 2023-10-30T12:51:47.412 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/38' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/28' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/24' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/29' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/32' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/22' 2023-10-30T12:51:47.413 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/25' 2023-10-30T12:51:47.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/30' 2023-10-30T12:51:47.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/33' 2023-10-30T12:51:47.414 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/23' 2023-10-30T12:51:48.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/35' 2023-10-30T12:51:48.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/26' 2023-10-30T12:51:48.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './266/37' 2023-10-30T12:51:48.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './266' 2023-10-30T12:51:48.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/21' 2023-10-30T12:51:48.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/27' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/31' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/36' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/20' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/39' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/34' 2023-10-30T12:51:48.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/38' 2023-10-30T12:51:48.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/28' 2023-10-30T12:51:48.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/24' 2023-10-30T12:51:48.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/29' 2023-10-30T12:51:48.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/32' 2023-10-30T12:51:48.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/22' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/25' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/30' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/33' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/23' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/35' 2023-10-30T12:51:48.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/26' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './267/37' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './267' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/21' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/27' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/31' 2023-10-30T12:51:48.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/36' 2023-10-30T12:51:48.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/20' 2023-10-30T12:51:48.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/39' 2023-10-30T12:51:48.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/34' 2023-10-30T12:51:48.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/38' 2023-10-30T12:51:48.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/28' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/24' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/29' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/32' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/22' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/25' 2023-10-30T12:51:48.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/30' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/33' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/23' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/35' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/26' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './268/37' 2023-10-30T12:51:48.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './268' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/21' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/27' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/31' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/36' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/20' 2023-10-30T12:51:48.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/39' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/34' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/38' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/28' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/24' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/29' 2023-10-30T12:51:48.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/32' 2023-10-30T12:51:48.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/22' 2023-10-30T12:51:48.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/25' 2023-10-30T12:51:48.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/30' 2023-10-30T12:51:48.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/33' 2023-10-30T12:51:48.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/23' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/35' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/26' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './269/37' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './269' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/21' 2023-10-30T12:51:48.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/27' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/31' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/36' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/20' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/39' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/34' 2023-10-30T12:51:48.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/38' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/28' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/24' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/29' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/32' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/22' 2023-10-30T12:51:48.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/25' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/30' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/33' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/23' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/35' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/26' 2023-10-30T12:51:48.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './270/37' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './270' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/21' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/27' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/31' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/36' 2023-10-30T12:51:48.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/20' 2023-10-30T12:51:48.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/39' 2023-10-30T12:51:48.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/34' 2023-10-30T12:51:48.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/38' 2023-10-30T12:51:48.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/28' 2023-10-30T12:51:48.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/24' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/29' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/32' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/22' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/25' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/30' 2023-10-30T12:51:48.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/33' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/23' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/35' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/26' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './271/37' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './271' 2023-10-30T12:51:48.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/21' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/27' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/31' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/36' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/20' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/39' 2023-10-30T12:51:48.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/34' 2023-10-30T12:51:48.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/38' 2023-10-30T12:51:48.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/28' 2023-10-30T12:51:48.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/24' 2023-10-30T12:51:48.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/29' 2023-10-30T12:51:48.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/32' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/22' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/25' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/30' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/33' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/23' 2023-10-30T12:51:48.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/35' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/26' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './272/37' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './272' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/21' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/27' 2023-10-30T12:51:48.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/31' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/36' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/20' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/39' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/34' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/38' 2023-10-30T12:51:48.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/28' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/24' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/29' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/32' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/22' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/25' 2023-10-30T12:51:48.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/30' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/33' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/23' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/35' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/26' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './273/37' 2023-10-30T12:51:48.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './273' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/21' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/27' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/31' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/36' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/20' 2023-10-30T12:51:48.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/39' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/34' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/38' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/28' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/24' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/29' 2023-10-30T12:51:48.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/32' 2023-10-30T12:51:48.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/22' 2023-10-30T12:51:48.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/25' 2023-10-30T12:51:48.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/30' 2023-10-30T12:51:48.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/33' 2023-10-30T12:51:48.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/23' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/35' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/26' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './274/37' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './274' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/21' 2023-10-30T12:51:48.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/27' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/31' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/36' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/20' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/39' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/34' 2023-10-30T12:51:48.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/38' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/28' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/24' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/29' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/32' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/22' 2023-10-30T12:51:48.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/25' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/30' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/33' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/23' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/35' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/26' 2023-10-30T12:51:48.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './275/37' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './275' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/21' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/27' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/31' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/36' 2023-10-30T12:51:48.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/20' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/39' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/34' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/38' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/28' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/24' 2023-10-30T12:51:48.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/29' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/32' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/22' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/25' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/30' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/33' 2023-10-30T12:51:48.841 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/23' 2023-10-30T12:51:48.842 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/35' 2023-10-30T12:51:48.842 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/26' 2023-10-30T12:51:54.257 INFO:tasks.workunit.client.0.smithi008.stdout:removed './276/37' 2023-10-30T12:51:54.257 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './276' 2023-10-30T12:51:54.257 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/21' 2023-10-30T12:51:54.257 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/27' 2023-10-30T12:51:54.258 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/31' 2023-10-30T12:51:54.258 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/36' 2023-10-30T12:51:54.258 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/20' 2023-10-30T12:51:54.258 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/39' 2023-10-30T12:51:54.258 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/34' 2023-10-30T12:51:54.259 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/38' 2023-10-30T12:51:54.259 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/28' 2023-10-30T12:51:54.259 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/24' 2023-10-30T12:51:54.259 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/29' 2023-10-30T12:51:54.259 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/32' 2023-10-30T12:51:54.260 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/22' 2023-10-30T12:51:54.260 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/25' 2023-10-30T12:51:54.260 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/30' 2023-10-30T12:51:54.260 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/33' 2023-10-30T12:51:54.260 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/23' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/35' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/26' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed './277/37' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './277' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/21' 2023-10-30T12:51:54.261 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/27' 2023-10-30T12:51:54.262 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/31' 2023-10-30T12:51:54.262 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/36' 2023-10-30T12:51:54.262 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/20' 2023-10-30T12:51:54.262 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/39' 2023-10-30T12:51:54.263 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/34' 2023-10-30T12:51:54.263 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/38' 2023-10-30T12:51:54.263 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/28' 2023-10-30T12:51:54.263 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/24' 2023-10-30T12:51:54.263 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/29' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/32' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/22' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/25' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/30' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/33' 2023-10-30T12:51:54.264 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/23' 2023-10-30T12:51:54.265 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/35' 2023-10-30T12:51:54.265 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/26' 2023-10-30T12:51:54.265 INFO:tasks.workunit.client.0.smithi008.stdout:removed './278/37' 2023-10-30T12:51:54.265 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './278' 2023-10-30T12:51:54.265 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/21' 2023-10-30T12:51:54.266 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/27' 2023-10-30T12:51:54.266 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/31' 2023-10-30T12:51:54.266 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/36' 2023-10-30T12:51:54.266 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/20' 2023-10-30T12:51:54.266 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/39' 2023-10-30T12:51:54.267 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/34' 2023-10-30T12:51:54.267 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/38' 2023-10-30T12:51:54.267 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/28' 2023-10-30T12:51:54.267 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/24' 2023-10-30T12:51:54.267 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/29' 2023-10-30T12:51:54.268 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/32' 2023-10-30T12:51:54.268 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/22' 2023-10-30T12:51:54.268 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/25' 2023-10-30T12:51:54.268 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/30' 2023-10-30T12:51:54.268 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/33' 2023-10-30T12:51:54.269 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/23' 2023-10-30T12:51:54.269 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/35' 2023-10-30T12:51:54.269 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/26' 2023-10-30T12:51:54.269 INFO:tasks.workunit.client.0.smithi008.stdout:removed './279/37' 2023-10-30T12:51:54.269 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './279' 2023-10-30T12:51:54.270 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/21' 2023-10-30T12:51:54.270 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/27' 2023-10-30T12:51:54.270 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/31' 2023-10-30T12:51:54.270 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/36' 2023-10-30T12:51:54.270 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/20' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/39' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/34' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/38' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/28' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/24' 2023-10-30T12:51:54.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/29' 2023-10-30T12:51:54.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/32' 2023-10-30T12:51:54.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/22' 2023-10-30T12:51:54.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/25' 2023-10-30T12:51:54.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/30' 2023-10-30T12:51:54.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/33' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/23' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/35' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/26' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './280/37' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './280' 2023-10-30T12:51:54.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/21' 2023-10-30T12:51:54.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/27' 2023-10-30T12:51:54.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/31' 2023-10-30T12:51:54.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/36' 2023-10-30T12:51:54.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/20' 2023-10-30T12:51:54.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/39' 2023-10-30T12:51:54.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/34' 2023-10-30T12:51:54.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/38' 2023-10-30T12:51:54.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/28' 2023-10-30T12:51:54.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/24' 2023-10-30T12:51:54.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/29' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/32' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/22' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/25' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/30' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/33' 2023-10-30T12:51:54.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/23' 2023-10-30T12:51:54.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/35' 2023-10-30T12:51:54.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/26' 2023-10-30T12:51:54.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './281/37' 2023-10-30T12:51:54.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './281' 2023-10-30T12:51:54.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/21' 2023-10-30T12:51:54.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/27' 2023-10-30T12:51:54.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/31' 2023-10-30T12:51:54.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/36' 2023-10-30T12:51:54.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/20' 2023-10-30T12:51:54.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/39' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/34' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/38' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/28' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/24' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/29' 2023-10-30T12:51:54.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/32' 2023-10-30T12:51:54.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/22' 2023-10-30T12:51:54.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/25' 2023-10-30T12:51:54.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/30' 2023-10-30T12:51:54.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/33' 2023-10-30T12:51:54.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/23' 2023-10-30T12:51:54.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/35' 2023-10-30T12:51:54.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/26' 2023-10-30T12:51:54.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './282/37' 2023-10-30T12:51:54.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './282' 2023-10-30T12:51:54.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/21' 2023-10-30T12:51:54.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/27' 2023-10-30T12:51:54.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/31' 2023-10-30T12:51:54.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/36' 2023-10-30T12:51:54.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/20' 2023-10-30T12:51:54.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/39' 2023-10-30T12:51:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/34' 2023-10-30T12:51:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/38' 2023-10-30T12:51:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/28' 2023-10-30T12:51:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/24' 2023-10-30T12:51:54.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/29' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/32' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/22' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/25' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/30' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/33' 2023-10-30T12:51:54.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/23' 2023-10-30T12:51:54.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/35' 2023-10-30T12:51:54.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/26' 2023-10-30T12:51:54.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './283/37' 2023-10-30T12:51:54.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './283' 2023-10-30T12:51:54.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/21' 2023-10-30T12:51:54.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/27' 2023-10-30T12:51:54.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/31' 2023-10-30T12:51:54.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/36' 2023-10-30T12:51:54.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/20' 2023-10-30T12:51:54.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/39' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/34' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/38' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/28' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/24' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/29' 2023-10-30T12:51:54.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/32' 2023-10-30T12:51:54.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/22' 2023-10-30T12:51:54.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/25' 2023-10-30T12:51:54.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/30' 2023-10-30T12:51:54.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/33' 2023-10-30T12:51:54.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/23' 2023-10-30T12:51:54.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/35' 2023-10-30T12:51:54.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/26' 2023-10-30T12:51:54.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './284/37' 2023-10-30T12:51:54.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './284' 2023-10-30T12:51:54.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/21' 2023-10-30T12:51:54.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/27' 2023-10-30T12:51:54.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/31' 2023-10-30T12:51:54.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/36' 2023-10-30T12:51:54.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/20' 2023-10-30T12:51:54.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/39' 2023-10-30T12:51:54.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/34' 2023-10-30T12:51:54.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/38' 2023-10-30T12:51:54.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/28' 2023-10-30T12:51:54.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/24' 2023-10-30T12:51:54.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/29' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/32' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/22' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/25' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/30' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/33' 2023-10-30T12:51:54.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/23' 2023-10-30T12:51:54.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/35' 2023-10-30T12:51:54.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/26' 2023-10-30T12:51:54.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './285/37' 2023-10-30T12:51:54.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './285' 2023-10-30T12:51:54.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/21' 2023-10-30T12:51:54.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/27' 2023-10-30T12:51:54.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/31' 2023-10-30T12:51:54.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/36' 2023-10-30T12:51:54.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/20' 2023-10-30T12:51:54.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/39' 2023-10-30T12:51:54.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/34' 2023-10-30T12:51:54.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/38' 2023-10-30T12:51:54.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/28' 2023-10-30T12:51:54.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/24' 2023-10-30T12:51:54.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/29' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/32' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/22' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/25' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/30' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/33' 2023-10-30T12:51:54.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/23' 2023-10-30T12:51:54.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/35' 2023-10-30T12:51:54.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/26' 2023-10-30T12:51:54.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './286/37' 2023-10-30T12:51:54.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './286' 2023-10-30T12:51:54.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/21' 2023-10-30T12:51:54.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/27' 2023-10-30T12:51:54.977 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/31' 2023-10-30T12:51:54.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/36' 2023-10-30T12:51:54.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/20' 2023-10-30T12:51:54.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/39' 2023-10-30T12:51:54.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/34' 2023-10-30T12:51:54.978 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/38' 2023-10-30T12:51:54.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/28' 2023-10-30T12:51:54.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/24' 2023-10-30T12:51:54.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/29' 2023-10-30T12:51:54.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/32' 2023-10-30T12:51:54.979 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/22' 2023-10-30T12:51:54.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/25' 2023-10-30T12:51:54.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/30' 2023-10-30T12:51:54.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/33' 2023-10-30T12:51:54.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/23' 2023-10-30T12:51:54.980 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/35' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/26' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './287/37' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './287' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/21' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/27' 2023-10-30T12:51:54.981 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/31' 2023-10-30T12:51:54.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/36' 2023-10-30T12:51:54.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/20' 2023-10-30T12:51:54.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/39' 2023-10-30T12:51:54.982 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/34' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/38' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/28' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/24' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/29' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/32' 2023-10-30T12:51:54.983 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/22' 2023-10-30T12:51:54.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/25' 2023-10-30T12:51:54.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/30' 2023-10-30T12:51:54.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/33' 2023-10-30T12:51:54.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/23' 2023-10-30T12:51:54.984 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/35' 2023-10-30T12:51:54.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/26' 2023-10-30T12:51:54.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './288/37' 2023-10-30T12:51:54.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './288' 2023-10-30T12:51:54.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/21' 2023-10-30T12:51:54.985 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/27' 2023-10-30T12:51:54.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/31' 2023-10-30T12:51:54.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/36' 2023-10-30T12:51:54.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/20' 2023-10-30T12:51:54.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/39' 2023-10-30T12:51:54.986 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/34' 2023-10-30T12:51:54.987 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/38' 2023-10-30T12:51:54.987 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/28' 2023-10-30T12:51:54.987 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/24' 2023-10-30T12:51:54.987 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/29' 2023-10-30T12:51:54.987 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/32' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/22' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/25' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/30' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/33' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/23' 2023-10-30T12:51:54.988 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/35' 2023-10-30T12:51:54.989 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/26' 2023-10-30T12:51:54.989 INFO:tasks.workunit.client.0.smithi008.stdout:removed './289/37' 2023-10-30T12:51:54.989 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './289' 2023-10-30T12:51:54.989 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/21' 2023-10-30T12:51:54.989 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/27' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/31' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/36' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/20' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/39' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/34' 2023-10-30T12:51:54.990 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/38' 2023-10-30T12:51:54.991 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/28' 2023-10-30T12:51:54.991 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/24' 2023-10-30T12:51:54.991 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/29' 2023-10-30T12:51:54.991 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/32' 2023-10-30T12:51:54.992 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/22' 2023-10-30T12:51:54.992 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/25' 2023-10-30T12:51:54.992 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/30' 2023-10-30T12:51:54.992 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/33' 2023-10-30T12:51:54.992 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/23' 2023-10-30T12:51:54.993 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/35' 2023-10-30T12:51:54.993 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/26' 2023-10-30T12:51:54.993 INFO:tasks.workunit.client.0.smithi008.stdout:removed './290/37' 2023-10-30T12:51:54.993 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './290' 2023-10-30T12:51:54.993 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/21' 2023-10-30T12:51:54.994 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/27' 2023-10-30T12:51:54.994 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/31' 2023-10-30T12:51:54.994 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/36' 2023-10-30T12:51:54.994 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/20' 2023-10-30T12:51:54.994 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/39' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/34' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/38' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/28' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/24' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/29' 2023-10-30T12:51:54.995 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/32' 2023-10-30T12:51:54.996 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/22' 2023-10-30T12:51:54.996 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/25' 2023-10-30T12:51:54.996 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/30' 2023-10-30T12:51:54.996 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/33' 2023-10-30T12:51:54.996 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/23' 2023-10-30T12:51:54.997 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/35' 2023-10-30T12:51:54.997 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/26' 2023-10-30T12:51:54.997 INFO:tasks.workunit.client.0.smithi008.stdout:removed './291/37' 2023-10-30T12:51:54.997 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './291' 2023-10-30T12:51:54.997 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/21' 2023-10-30T12:51:54.998 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/27' 2023-10-30T12:51:54.998 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/31' 2023-10-30T12:51:54.998 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/36' 2023-10-30T12:51:54.998 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/20' 2023-10-30T12:51:54.998 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/39' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/34' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/38' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/28' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/24' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/29' 2023-10-30T12:51:54.999 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/32' 2023-10-30T12:51:55.000 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/22' 2023-10-30T12:51:55.000 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/25' 2023-10-30T12:51:55.000 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/30' 2023-10-30T12:51:55.000 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/33' 2023-10-30T12:51:55.000 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/23' 2023-10-30T12:51:55.001 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/35' 2023-10-30T12:51:55.001 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/26' 2023-10-30T12:51:55.001 INFO:tasks.workunit.client.0.smithi008.stdout:removed './292/37' 2023-10-30T12:51:55.001 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './292' 2023-10-30T12:51:55.001 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/21' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/27' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/31' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/36' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/20' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/39' 2023-10-30T12:51:55.002 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/34' 2023-10-30T12:51:55.003 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/38' 2023-10-30T12:51:55.003 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/28' 2023-10-30T12:51:55.003 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/24' 2023-10-30T12:51:55.003 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/29' 2023-10-30T12:51:55.003 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/32' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/22' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/25' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/30' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/33' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/23' 2023-10-30T12:51:55.004 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/35' 2023-10-30T12:51:55.005 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/26' 2023-10-30T12:51:55.005 INFO:tasks.workunit.client.0.smithi008.stdout:removed './293/37' 2023-10-30T12:51:55.005 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './293' 2023-10-30T12:51:55.005 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/21' 2023-10-30T12:51:55.005 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/27' 2023-10-30T12:51:55.006 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/31' 2023-10-30T12:51:55.006 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/36' 2023-10-30T12:51:55.006 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/20' 2023-10-30T12:51:55.006 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/39' 2023-10-30T12:51:55.006 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/34' 2023-10-30T12:51:55.007 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/38' 2023-10-30T12:51:55.007 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/28' 2023-10-30T12:51:55.007 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/24' 2023-10-30T12:51:55.008 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/29' 2023-10-30T12:51:55.008 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/32' 2023-10-30T12:51:55.008 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/22' 2023-10-30T12:51:55.009 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/25' 2023-10-30T12:51:55.009 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/30' 2023-10-30T12:51:55.009 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/33' 2023-10-30T12:51:55.010 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/23' 2023-10-30T12:51:55.010 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/35' 2023-10-30T12:51:55.010 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/26' 2023-10-30T12:51:55.011 INFO:tasks.workunit.client.0.smithi008.stdout:removed './294/37' 2023-10-30T12:51:55.011 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './294' 2023-10-30T12:51:55.011 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/21' 2023-10-30T12:51:55.012 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/27' 2023-10-30T12:51:55.012 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/31' 2023-10-30T12:51:55.012 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/36' 2023-10-30T12:51:55.013 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/20' 2023-10-30T12:51:55.013 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/39' 2023-10-30T12:51:55.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/34' 2023-10-30T12:51:55.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/38' 2023-10-30T12:51:55.014 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/28' 2023-10-30T12:51:55.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/24' 2023-10-30T12:51:55.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/29' 2023-10-30T12:51:55.015 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/32' 2023-10-30T12:51:55.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/22' 2023-10-30T12:51:55.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/25' 2023-10-30T12:51:55.016 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/30' 2023-10-30T12:51:55.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/33' 2023-10-30T12:51:55.017 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/23' 2023-10-30T12:51:55.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/35' 2023-10-30T12:51:55.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/26' 2023-10-30T12:51:55.018 INFO:tasks.workunit.client.0.smithi008.stdout:removed './295/37' 2023-10-30T12:51:55.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './295' 2023-10-30T12:51:55.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/21' 2023-10-30T12:51:55.019 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/27' 2023-10-30T12:51:55.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/31' 2023-10-30T12:51:55.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/36' 2023-10-30T12:51:55.020 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/20' 2023-10-30T12:51:55.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/39' 2023-10-30T12:51:55.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/34' 2023-10-30T12:51:55.021 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/38' 2023-10-30T12:51:55.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/28' 2023-10-30T12:51:55.022 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/24' 2023-10-30T12:51:55.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/29' 2023-10-30T12:51:55.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/32' 2023-10-30T12:51:55.023 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/22' 2023-10-30T12:51:55.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/25' 2023-10-30T12:51:55.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/30' 2023-10-30T12:51:55.024 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/33' 2023-10-30T12:51:55.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/23' 2023-10-30T12:51:55.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/35' 2023-10-30T12:51:55.025 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/26' 2023-10-30T12:51:55.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './296/37' 2023-10-30T12:51:55.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './296' 2023-10-30T12:51:55.026 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/21' 2023-10-30T12:51:55.027 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/27' 2023-10-30T12:52:03.271 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/31' 2023-10-30T12:52:03.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/36' 2023-10-30T12:52:03.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/20' 2023-10-30T12:52:03.272 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/39' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/34' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/38' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/28' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/24' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/29' 2023-10-30T12:52:03.273 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/32' 2023-10-30T12:52:03.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/22' 2023-10-30T12:52:03.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/25' 2023-10-30T12:52:03.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/30' 2023-10-30T12:52:03.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/33' 2023-10-30T12:52:03.274 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/23' 2023-10-30T12:52:03.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/35' 2023-10-30T12:52:03.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/26' 2023-10-30T12:52:03.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './297/37' 2023-10-30T12:52:03.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './297' 2023-10-30T12:52:03.275 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/21' 2023-10-30T12:52:03.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/27' 2023-10-30T12:52:03.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/31' 2023-10-30T12:52:03.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/36' 2023-10-30T12:52:03.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/20' 2023-10-30T12:52:03.276 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/39' 2023-10-30T12:52:03.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/34' 2023-10-30T12:52:03.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/38' 2023-10-30T12:52:03.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/28' 2023-10-30T12:52:03.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/24' 2023-10-30T12:52:03.277 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/29' 2023-10-30T12:52:03.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/32' 2023-10-30T12:52:03.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/22' 2023-10-30T12:52:03.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/25' 2023-10-30T12:52:03.278 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/30' 2023-10-30T12:52:03.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/33' 2023-10-30T12:52:03.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/23' 2023-10-30T12:52:03.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/35' 2023-10-30T12:52:03.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/26' 2023-10-30T12:52:03.279 INFO:tasks.workunit.client.0.smithi008.stdout:removed './298/37' 2023-10-30T12:52:03.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './298' 2023-10-30T12:52:03.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/21' 2023-10-30T12:52:03.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/27' 2023-10-30T12:52:03.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/31' 2023-10-30T12:52:03.280 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/36' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/20' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/39' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/34' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/38' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/28' 2023-10-30T12:52:03.281 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/24' 2023-10-30T12:52:03.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/29' 2023-10-30T12:52:03.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/32' 2023-10-30T12:52:03.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/22' 2023-10-30T12:52:03.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/25' 2023-10-30T12:52:03.282 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/30' 2023-10-30T12:52:03.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/33' 2023-10-30T12:52:03.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/23' 2023-10-30T12:52:03.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/35' 2023-10-30T12:52:03.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/26' 2023-10-30T12:52:03.283 INFO:tasks.workunit.client.0.smithi008.stdout:removed './299/37' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './299' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/19' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/15' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/20' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/12' 2023-10-30T12:52:03.284 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/10' 2023-10-30T12:52:03.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/11' 2023-10-30T12:52:03.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/16' 2023-10-30T12:52:03.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/14' 2023-10-30T12:52:03.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/18' 2023-10-30T12:52:03.285 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/13' 2023-10-30T12:52:03.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/166/17' 2023-10-30T12:52:03.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/166' 2023-10-30T12:52:03.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/19' 2023-10-30T12:52:03.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/15' 2023-10-30T12:52:03.286 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/20' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/12' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/10' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/11' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/16' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/14' 2023-10-30T12:52:03.287 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/18' 2023-10-30T12:52:03.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/13' 2023-10-30T12:52:03.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/147/17' 2023-10-30T12:52:03.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/147' 2023-10-30T12:52:03.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/19' 2023-10-30T12:52:03.288 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/15' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/20' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/12' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/10' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/11' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/16' 2023-10-30T12:52:03.289 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/14' 2023-10-30T12:52:03.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/18' 2023-10-30T12:52:03.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/13' 2023-10-30T12:52:03.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/151/17' 2023-10-30T12:52:03.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/151' 2023-10-30T12:52:03.290 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/19' 2023-10-30T12:52:03.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/15' 2023-10-30T12:52:03.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/20' 2023-10-30T12:52:03.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/12' 2023-10-30T12:52:03.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/10' 2023-10-30T12:52:03.291 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/11' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/16' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/14' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/18' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/13' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/170/17' 2023-10-30T12:52:03.292 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/170' 2023-10-30T12:52:03.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/19' 2023-10-30T12:52:03.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/15' 2023-10-30T12:52:03.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/20' 2023-10-30T12:52:03.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/12' 2023-10-30T12:52:03.293 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/10' 2023-10-30T12:52:03.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/11' 2023-10-30T12:52:03.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/16' 2023-10-30T12:52:03.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/14' 2023-10-30T12:52:03.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/18' 2023-10-30T12:52:03.294 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/13' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/167/17' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/167' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/19' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/15' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/20' 2023-10-30T12:52:03.295 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/12' 2023-10-30T12:52:03.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/10' 2023-10-30T12:52:03.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/11' 2023-10-30T12:52:03.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/16' 2023-10-30T12:52:03.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/14' 2023-10-30T12:52:03.296 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/18' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/13' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/140/17' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/140' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/19' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/15' 2023-10-30T12:52:03.297 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/20' 2023-10-30T12:52:03.298 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/12' 2023-10-30T12:52:03.298 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/10' 2023-10-30T12:52:03.803 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/11' 2023-10-30T12:52:03.803 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/16' 2023-10-30T12:52:03.804 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/14' 2023-10-30T12:52:03.804 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/18' 2023-10-30T12:52:03.804 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/13' 2023-10-30T12:52:03.804 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/142/17' 2023-10-30T12:52:03.804 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/142' 2023-10-30T12:52:03.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/19' 2023-10-30T12:52:03.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/15' 2023-10-30T12:52:03.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/20' 2023-10-30T12:52:03.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/12' 2023-10-30T12:52:03.805 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/10' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/11' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/16' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/14' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/18' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/13' 2023-10-30T12:52:03.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/150/17' 2023-10-30T12:52:03.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/150' 2023-10-30T12:52:03.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/19' 2023-10-30T12:52:03.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/15' 2023-10-30T12:52:03.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/20' 2023-10-30T12:52:03.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/12' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/10' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/11' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/16' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/14' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/18' 2023-10-30T12:52:03.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/13' 2023-10-30T12:52:03.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/105/17' 2023-10-30T12:52:03.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/105' 2023-10-30T12:52:03.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/19' 2023-10-30T12:52:03.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/15' 2023-10-30T12:52:03.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/20' 2023-10-30T12:52:03.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/12' 2023-10-30T12:52:03.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/10' 2023-10-30T12:52:03.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/11' 2023-10-30T12:52:03.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/16' 2023-10-30T12:52:03.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/14' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/18' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/13' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/149/17' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/149' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/19' 2023-10-30T12:52:03.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/15' 2023-10-30T12:52:03.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/20' 2023-10-30T12:52:03.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/12' 2023-10-30T12:52:03.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/10' 2023-10-30T12:52:03.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/11' 2023-10-30T12:52:03.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/16' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/14' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/18' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/13' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/109/17' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/109' 2023-10-30T12:52:03.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/19' 2023-10-30T12:52:03.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/15' 2023-10-30T12:52:03.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/20' 2023-10-30T12:52:03.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/12' 2023-10-30T12:52:03.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/10' 2023-10-30T12:52:03.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/11' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/16' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/14' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/18' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/13' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/158/17' 2023-10-30T12:52:03.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/158' 2023-10-30T12:52:03.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/19' 2023-10-30T12:52:03.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/15' 2023-10-30T12:52:03.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/20' 2023-10-30T12:52:03.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/12' 2023-10-30T12:52:03.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/10' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/11' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/16' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/14' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/18' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/13' 2023-10-30T12:52:03.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/106/17' 2023-10-30T12:52:03.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/106' 2023-10-30T12:52:03.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/19' 2023-10-30T12:52:03.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/15' 2023-10-30T12:52:03.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/20' 2023-10-30T12:52:03.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/12' 2023-10-30T12:52:03.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/10' 2023-10-30T12:52:03.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/11' 2023-10-30T12:52:03.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/16' 2023-10-30T12:52:03.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/14' 2023-10-30T12:52:03.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/18' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/13' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/189/17' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/189' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/19' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/15' 2023-10-30T12:52:03.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/20' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/12' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/10' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/11' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/16' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/14' 2023-10-30T12:52:03.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/18' 2023-10-30T12:52:03.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/13' 2023-10-30T12:52:03.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/133/17' 2023-10-30T12:52:03.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/133' 2023-10-30T12:52:03.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/19' 2023-10-30T12:52:03.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/15' 2023-10-30T12:52:03.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/20' 2023-10-30T12:52:03.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/12' 2023-10-30T12:52:03.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/10' 2023-10-30T12:52:03.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/11' 2023-10-30T12:52:08.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/16' 2023-10-30T12:52:08.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/14' 2023-10-30T12:52:08.146 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/18' 2023-10-30T12:52:08.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/13' 2023-10-30T12:52:08.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/173/17' 2023-10-30T12:52:08.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/173' 2023-10-30T12:52:08.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/19' 2023-10-30T12:52:08.147 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/15' 2023-10-30T12:52:08.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/20' 2023-10-30T12:52:08.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/12' 2023-10-30T12:52:08.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/10' 2023-10-30T12:52:08.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/11' 2023-10-30T12:52:08.148 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/16' 2023-10-30T12:52:08.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/14' 2023-10-30T12:52:08.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/18' 2023-10-30T12:52:08.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/13' 2023-10-30T12:52:08.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/174/17' 2023-10-30T12:52:08.149 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/174' 2023-10-30T12:52:08.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/19' 2023-10-30T12:52:08.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/15' 2023-10-30T12:52:08.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/20' 2023-10-30T12:52:08.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/12' 2023-10-30T12:52:08.150 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/10' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/11' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/16' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/14' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/18' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/13' 2023-10-30T12:52:08.151 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/155/17' 2023-10-30T12:52:08.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/155' 2023-10-30T12:52:08.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/19' 2023-10-30T12:52:08.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/15' 2023-10-30T12:52:08.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/20' 2023-10-30T12:52:08.152 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/12' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/10' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/11' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/16' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/14' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/18' 2023-10-30T12:52:08.153 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/13' 2023-10-30T12:52:08.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/114/17' 2023-10-30T12:52:08.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/114' 2023-10-30T12:52:08.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/19' 2023-10-30T12:52:08.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/15' 2023-10-30T12:52:08.154 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/20' 2023-10-30T12:52:08.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/12' 2023-10-30T12:52:08.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/10' 2023-10-30T12:52:08.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/11' 2023-10-30T12:52:08.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/16' 2023-10-30T12:52:08.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/14' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/18' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/13' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/159/17' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/159' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/19' 2023-10-30T12:52:08.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/15' 2023-10-30T12:52:08.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/20' 2023-10-30T12:52:08.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/12' 2023-10-30T12:52:08.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/10' 2023-10-30T12:52:08.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/11' 2023-10-30T12:52:08.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/16' 2023-10-30T12:52:08.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/14' 2023-10-30T12:52:08.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/18' 2023-10-30T12:52:08.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/13' 2023-10-30T12:52:08.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/187/17' 2023-10-30T12:52:08.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/187' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/19' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/15' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/20' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/12' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/10' 2023-10-30T12:52:08.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/11' 2023-10-30T12:52:08.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/16' 2023-10-30T12:52:08.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/14' 2023-10-30T12:52:08.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/18' 2023-10-30T12:52:08.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/13' 2023-10-30T12:52:08.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/185/17' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/185' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/19' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/15' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/20' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/12' 2023-10-30T12:52:08.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/10' 2023-10-30T12:52:08.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/11' 2023-10-30T12:52:08.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/16' 2023-10-30T12:52:08.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/14' 2023-10-30T12:52:08.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/18' 2023-10-30T12:52:08.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/13' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/120/17' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/120' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/19' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/15' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/20' 2023-10-30T12:52:08.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/12' 2023-10-30T12:52:08.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/10' 2023-10-30T12:52:08.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/11' 2023-10-30T12:52:08.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/16' 2023-10-30T12:52:08.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/14' 2023-10-30T12:52:08.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/18' 2023-10-30T12:52:08.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/13' 2023-10-30T12:52:08.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/110/17' 2023-10-30T12:52:08.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/110' 2023-10-30T12:52:08.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/19' 2023-10-30T12:52:08.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/15' 2023-10-30T12:52:08.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/20' 2023-10-30T12:52:08.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/12' 2023-10-30T12:52:08.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/10' 2023-10-30T12:52:08.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/11' 2023-10-30T12:52:08.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/16' 2023-10-30T12:52:09.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/14' 2023-10-30T12:52:09.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/18' 2023-10-30T12:52:09.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/13' 2023-10-30T12:52:09.563 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/137/17' 2023-10-30T12:52:09.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/137' 2023-10-30T12:52:09.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/19' 2023-10-30T12:52:09.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/15' 2023-10-30T12:52:09.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/20' 2023-10-30T12:52:09.564 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/12' 2023-10-30T12:52:09.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/10' 2023-10-30T12:52:09.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/11' 2023-10-30T12:52:09.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/16' 2023-10-30T12:52:09.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/14' 2023-10-30T12:52:09.565 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/18' 2023-10-30T12:52:09.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/13' 2023-10-30T12:52:09.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/198/17' 2023-10-30T12:52:09.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/198' 2023-10-30T12:52:09.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/19' 2023-10-30T12:52:09.566 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/15' 2023-10-30T12:52:09.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/20' 2023-10-30T12:52:09.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/12' 2023-10-30T12:52:09.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/10' 2023-10-30T12:52:09.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/11' 2023-10-30T12:52:09.567 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/16' 2023-10-30T12:52:09.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/14' 2023-10-30T12:52:09.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/18' 2023-10-30T12:52:09.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/13' 2023-10-30T12:52:09.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/136/17' 2023-10-30T12:52:09.568 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/136' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/19' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/15' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/20' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/12' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/10' 2023-10-30T12:52:09.569 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/11' 2023-10-30T12:52:09.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/16' 2023-10-30T12:52:09.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/14' 2023-10-30T12:52:09.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/18' 2023-10-30T12:52:09.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/13' 2023-10-30T12:52:09.570 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/125/17' 2023-10-30T12:52:09.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/125' 2023-10-30T12:52:09.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/19' 2023-10-30T12:52:09.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/15' 2023-10-30T12:52:09.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/20' 2023-10-30T12:52:09.571 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/12' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/10' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/11' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/16' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/14' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/18' 2023-10-30T12:52:09.572 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/13' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/194/17' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/194' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/19' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/15' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/20' 2023-10-30T12:52:09.573 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/12' 2023-10-30T12:52:09.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/10' 2023-10-30T12:52:09.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/11' 2023-10-30T12:52:09.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/16' 2023-10-30T12:52:09.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/14' 2023-10-30T12:52:09.574 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/18' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/13' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/119/17' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/119' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/19' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/15' 2023-10-30T12:52:09.575 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/20' 2023-10-30T12:52:09.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/12' 2023-10-30T12:52:09.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/10' 2023-10-30T12:52:09.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/11' 2023-10-30T12:52:09.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/16' 2023-10-30T12:52:09.576 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/14' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/18' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/13' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/157/17' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/157' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/19' 2023-10-30T12:52:09.577 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/15' 2023-10-30T12:52:09.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/20' 2023-10-30T12:52:09.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/12' 2023-10-30T12:52:09.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/10' 2023-10-30T12:52:09.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/11' 2023-10-30T12:52:09.578 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/16' 2023-10-30T12:52:09.579 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/14' 2023-10-30T12:52:09.579 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/18' 2023-10-30T12:52:09.579 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/13' 2023-10-30T12:52:09.579 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/172/17' 2023-10-30T12:52:09.579 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/172' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/19' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/15' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/20' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/12' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/10' 2023-10-30T12:52:09.580 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/11' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/16' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/14' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/18' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/13' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/186/17' 2023-10-30T12:52:09.581 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/186' 2023-10-30T12:52:09.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/19' 2023-10-30T12:52:09.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/15' 2023-10-30T12:52:09.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/20' 2023-10-30T12:52:09.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/12' 2023-10-30T12:52:09.582 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/10' 2023-10-30T12:52:09.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/11' 2023-10-30T12:52:09.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/16' 2023-10-30T12:52:09.583 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/14' 2023-10-30T12:52:09.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/18' 2023-10-30T12:52:09.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/13' 2023-10-30T12:52:09.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/190/17' 2023-10-30T12:52:09.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/190' 2023-10-30T12:52:09.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/19' 2023-10-30T12:52:09.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/15' 2023-10-30T12:52:09.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/20' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/12' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/10' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/11' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/16' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/14' 2023-10-30T12:52:09.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/18' 2023-10-30T12:52:09.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/13' 2023-10-30T12:52:09.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/104/17' 2023-10-30T12:52:09.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/104' 2023-10-30T12:52:09.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/19' 2023-10-30T12:52:09.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/15' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/20' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/12' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/10' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/11' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/16' 2023-10-30T12:52:09.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/14' 2023-10-30T12:52:09.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/18' 2023-10-30T12:52:09.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/13' 2023-10-30T12:52:09.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/164/17' 2023-10-30T12:52:09.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/164' 2023-10-30T12:52:09.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/19' 2023-10-30T12:52:09.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/15' 2023-10-30T12:52:09.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/20' 2023-10-30T12:52:09.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/12' 2023-10-30T12:52:09.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/10' 2023-10-30T12:52:09.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/11' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/16' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/14' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/18' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/13' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/112/17' 2023-10-30T12:52:09.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/112' 2023-10-30T12:52:09.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/19' 2023-10-30T12:52:09.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/15' 2023-10-30T12:52:09.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/20' 2023-10-30T12:52:09.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/12' 2023-10-30T12:52:09.828 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/10' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/11' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/16' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/14' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/18' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/13' 2023-10-30T12:52:09.829 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/100/17' 2023-10-30T12:52:09.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/100' 2023-10-30T12:52:09.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/19' 2023-10-30T12:52:09.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/15' 2023-10-30T12:52:09.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/20' 2023-10-30T12:52:09.830 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/12' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/10' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/11' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/16' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/14' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/18' 2023-10-30T12:52:09.831 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/13' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/134/17' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/134' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/19' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/15' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/20' 2023-10-30T12:52:09.832 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/12' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/10' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/11' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/16' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/14' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/18' 2023-10-30T12:52:09.833 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/13' 2023-10-30T12:52:09.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/115/17' 2023-10-30T12:52:09.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/115' 2023-10-30T12:52:09.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/19' 2023-10-30T12:52:09.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/15' 2023-10-30T12:52:09.834 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/20' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/12' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/10' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/11' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/16' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/14' 2023-10-30T12:52:09.835 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/18' 2023-10-30T12:52:09.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/13' 2023-10-30T12:52:09.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/116/17' 2023-10-30T12:52:09.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/116' 2023-10-30T12:52:09.836 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/19' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/15' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/20' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/12' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/10' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/11' 2023-10-30T12:52:09.837 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/16' 2023-10-30T12:52:09.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/14' 2023-10-30T12:52:09.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/18' 2023-10-30T12:52:09.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/13' 2023-10-30T12:52:09.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/195/17' 2023-10-30T12:52:09.838 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/195' 2023-10-30T12:52:09.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/19' 2023-10-30T12:52:09.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/15' 2023-10-30T12:52:09.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/20' 2023-10-30T12:52:09.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/12' 2023-10-30T12:52:09.839 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/10' 2023-10-30T12:52:09.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/11' 2023-10-30T12:52:09.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/16' 2023-10-30T12:52:09.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/14' 2023-10-30T12:52:09.840 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/18' 2023-10-30T12:52:11.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/13' 2023-10-30T12:52:11.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/138/17' 2023-10-30T12:52:11.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/138' 2023-10-30T12:52:11.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/19' 2023-10-30T12:52:11.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/15' 2023-10-30T12:52:11.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/20' 2023-10-30T12:52:11.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/12' 2023-10-30T12:52:11.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/10' 2023-10-30T12:52:11.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/11' 2023-10-30T12:52:11.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/16' 2023-10-30T12:52:11.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/14' 2023-10-30T12:52:11.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/18' 2023-10-30T12:52:11.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/13' 2023-10-30T12:52:11.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/135/17' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/135' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/19' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/15' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/20' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/12' 2023-10-30T12:52:11.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/10' 2023-10-30T12:52:11.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/11' 2023-10-30T12:52:11.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/16' 2023-10-30T12:52:11.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/14' 2023-10-30T12:52:11.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/18' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/13' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/117/17' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/117' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/19' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/15' 2023-10-30T12:52:11.323 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/20' 2023-10-30T12:52:11.324 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/12' 2023-10-30T12:52:11.324 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/10' 2023-10-30T12:52:11.324 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/11' 2023-10-30T12:52:11.324 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/16' 2023-10-30T12:52:11.324 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/14' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/18' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/13' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/143/17' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/143' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/19' 2023-10-30T12:52:11.325 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/15' 2023-10-30T12:52:11.326 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/20' 2023-10-30T12:52:11.326 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/12' 2023-10-30T12:52:11.326 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/10' 2023-10-30T12:52:11.326 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/11' 2023-10-30T12:52:11.326 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/16' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/14' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/18' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/13' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/180/17' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/180' 2023-10-30T12:52:11.327 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/19' 2023-10-30T12:52:11.328 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/15' 2023-10-30T12:52:11.328 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/20' 2023-10-30T12:52:11.328 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/12' 2023-10-30T12:52:11.328 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/10' 2023-10-30T12:52:11.328 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/11' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/16' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/14' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/18' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/13' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/126/17' 2023-10-30T12:52:11.329 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/126' 2023-10-30T12:52:11.330 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/19' 2023-10-30T12:52:11.330 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/15' 2023-10-30T12:52:11.330 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/20' 2023-10-30T12:52:11.330 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/12' 2023-10-30T12:52:11.330 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/10' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/11' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/16' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/14' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/18' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/13' 2023-10-30T12:52:11.331 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/161/17' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/161' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/19' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/15' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/20' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/12' 2023-10-30T12:52:11.332 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/10' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/11' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/16' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/14' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/18' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/13' 2023-10-30T12:52:11.333 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/176/17' 2023-10-30T12:52:11.334 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/176' 2023-10-30T12:52:11.334 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/19' 2023-10-30T12:52:11.334 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/15' 2023-10-30T12:52:11.334 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/20' 2023-10-30T12:52:11.334 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/12' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/10' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/11' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/16' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/14' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/18' 2023-10-30T12:52:11.335 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/13' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/111/17' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/111' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/19' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/15' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/20' 2023-10-30T12:52:11.336 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/12' 2023-10-30T12:52:11.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/10' 2023-10-30T12:52:11.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/11' 2023-10-30T12:52:11.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/16' 2023-10-30T12:52:11.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/14' 2023-10-30T12:52:11.337 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/18' 2023-10-30T12:52:11.338 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/13' 2023-10-30T12:52:13.302 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/191/17' 2023-10-30T12:52:13.303 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/191' 2023-10-30T12:52:13.303 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/19' 2023-10-30T12:52:13.303 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/15' 2023-10-30T12:52:13.303 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/20' 2023-10-30T12:52:13.303 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/12' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/10' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/11' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/16' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/14' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/18' 2023-10-30T12:52:13.304 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/13' 2023-10-30T12:52:13.305 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/184/17' 2023-10-30T12:52:13.305 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/184' 2023-10-30T12:52:13.305 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/19' 2023-10-30T12:52:13.305 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/15' 2023-10-30T12:52:13.305 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/20' 2023-10-30T12:52:13.306 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/12' 2023-10-30T12:52:13.306 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/10' 2023-10-30T12:52:13.306 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/11' 2023-10-30T12:52:13.306 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/16' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/14' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/18' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/13' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/192/17' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/192' 2023-10-30T12:52:13.307 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/19' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/15' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/20' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/12' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/10' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/11' 2023-10-30T12:52:13.308 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/16' 2023-10-30T12:52:13.309 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/14' 2023-10-30T12:52:13.309 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/18' 2023-10-30T12:52:13.309 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/13' 2023-10-30T12:52:13.309 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/103/17' 2023-10-30T12:52:13.309 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/103' 2023-10-30T12:52:13.310 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/19' 2023-10-30T12:52:13.310 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/15' 2023-10-30T12:52:13.310 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/20' 2023-10-30T12:52:13.310 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/12' 2023-10-30T12:52:13.310 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/10' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/11' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/16' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/14' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/18' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/13' 2023-10-30T12:52:13.311 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/128/17' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/128' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/19' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/15' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/20' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/12' 2023-10-30T12:52:13.312 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/10' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/11' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/16' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/14' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/18' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/13' 2023-10-30T12:52:13.313 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/144/17' 2023-10-30T12:52:13.314 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/144' 2023-10-30T12:52:13.314 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/19' 2023-10-30T12:52:13.314 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/15' 2023-10-30T12:52:13.314 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/20' 2023-10-30T12:52:13.314 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/12' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/10' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/11' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/16' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/14' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/18' 2023-10-30T12:52:13.315 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/13' 2023-10-30T12:52:13.316 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/146/17' 2023-10-30T12:52:13.316 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/146' 2023-10-30T12:52:13.316 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/19' 2023-10-30T12:52:13.316 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/15' 2023-10-30T12:52:13.316 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/20' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/12' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/10' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/11' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/16' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/14' 2023-10-30T12:52:13.317 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/18' 2023-10-30T12:52:13.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/13' 2023-10-30T12:52:13.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/141/17' 2023-10-30T12:52:13.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/141' 2023-10-30T12:52:13.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/19' 2023-10-30T12:52:13.318 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/15' 2023-10-30T12:52:13.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/20' 2023-10-30T12:52:13.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/12' 2023-10-30T12:52:13.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/10' 2023-10-30T12:52:13.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/11' 2023-10-30T12:52:13.319 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/16' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/14' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/18' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/13' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/182/17' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/182' 2023-10-30T12:52:13.320 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/19' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/15' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/20' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/12' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/10' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/11' 2023-10-30T12:52:13.321 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/16' 2023-10-30T12:52:13.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/14' 2023-10-30T12:52:13.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/18' 2023-10-30T12:52:13.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/13' 2023-10-30T12:52:13.322 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/107/17' 2023-10-30T12:52:15.155 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/107' 2023-10-30T12:52:15.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/19' 2023-10-30T12:52:15.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/15' 2023-10-30T12:52:15.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/20' 2023-10-30T12:52:15.156 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/12' 2023-10-30T12:52:15.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/10' 2023-10-30T12:52:15.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/11' 2023-10-30T12:52:15.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/16' 2023-10-30T12:52:15.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/14' 2023-10-30T12:52:15.157 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/18' 2023-10-30T12:52:15.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/13' 2023-10-30T12:52:15.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/178/17' 2023-10-30T12:52:15.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/178' 2023-10-30T12:52:15.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/19' 2023-10-30T12:52:15.158 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/15' 2023-10-30T12:52:15.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/20' 2023-10-30T12:52:15.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/12' 2023-10-30T12:52:15.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/10' 2023-10-30T12:52:15.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/11' 2023-10-30T12:52:15.159 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/16' 2023-10-30T12:52:15.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/14' 2023-10-30T12:52:15.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/18' 2023-10-30T12:52:15.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/13' 2023-10-30T12:52:15.160 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/156/17' 2023-10-30T12:52:15.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/156' 2023-10-30T12:52:15.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/19' 2023-10-30T12:52:15.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/15' 2023-10-30T12:52:15.161 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/20' 2023-10-30T12:52:15.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/12' 2023-10-30T12:52:15.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/10' 2023-10-30T12:52:15.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/11' 2023-10-30T12:52:15.162 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/16' 2023-10-30T12:52:15.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/14' 2023-10-30T12:52:15.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/18' 2023-10-30T12:52:15.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/13' 2023-10-30T12:52:15.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/131/17' 2023-10-30T12:52:15.163 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/131' 2023-10-30T12:52:15.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/19' 2023-10-30T12:52:15.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/15' 2023-10-30T12:52:15.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/20' 2023-10-30T12:52:15.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/12' 2023-10-30T12:52:15.164 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/10' 2023-10-30T12:52:15.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/11' 2023-10-30T12:52:15.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/16' 2023-10-30T12:52:15.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/14' 2023-10-30T12:52:15.165 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/18' 2023-10-30T12:52:15.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/13' 2023-10-30T12:52:15.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/123/17' 2023-10-30T12:52:15.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/123' 2023-10-30T12:52:15.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/19' 2023-10-30T12:52:15.166 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/15' 2023-10-30T12:52:15.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/20' 2023-10-30T12:52:15.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/12' 2023-10-30T12:52:15.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/10' 2023-10-30T12:52:15.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/11' 2023-10-30T12:52:15.167 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/16' 2023-10-30T12:52:15.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/14' 2023-10-30T12:52:15.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/18' 2023-10-30T12:52:15.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/13' 2023-10-30T12:52:15.168 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/160/17' 2023-10-30T12:52:15.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/160' 2023-10-30T12:52:15.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/19' 2023-10-30T12:52:15.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/15' 2023-10-30T12:52:15.169 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/20' 2023-10-30T12:52:15.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/12' 2023-10-30T12:52:15.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/10' 2023-10-30T12:52:15.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/11' 2023-10-30T12:52:15.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/16' 2023-10-30T12:52:15.170 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/14' 2023-10-30T12:52:15.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/18' 2023-10-30T12:52:15.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/13' 2023-10-30T12:52:15.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/165/17' 2023-10-30T12:52:15.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/165' 2023-10-30T12:52:15.171 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/19' 2023-10-30T12:52:15.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/15' 2023-10-30T12:52:15.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/20' 2023-10-30T12:52:15.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/12' 2023-10-30T12:52:15.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/10' 2023-10-30T12:52:15.172 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/11' 2023-10-30T12:52:15.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/16' 2023-10-30T12:52:15.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/14' 2023-10-30T12:52:15.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/18' 2023-10-30T12:52:15.173 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/13' 2023-10-30T12:52:15.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/101/17' 2023-10-30T12:52:15.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/101' 2023-10-30T12:52:15.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/19' 2023-10-30T12:52:15.174 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/15' 2023-10-30T12:52:15.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/20' 2023-10-30T12:52:15.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/12' 2023-10-30T12:52:15.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/10' 2023-10-30T12:52:15.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/11' 2023-10-30T12:52:15.175 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/16' 2023-10-30T12:52:15.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/14' 2023-10-30T12:52:15.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/18' 2023-10-30T12:52:15.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/13' 2023-10-30T12:52:15.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/148/17' 2023-10-30T12:52:15.176 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/148' 2023-10-30T12:52:15.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/19' 2023-10-30T12:52:15.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/15' 2023-10-30T12:52:15.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/20' 2023-10-30T12:52:15.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/12' 2023-10-30T12:52:15.177 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/10' 2023-10-30T12:52:15.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/11' 2023-10-30T12:52:15.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/16' 2023-10-30T12:52:15.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/14' 2023-10-30T12:52:15.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/18' 2023-10-30T12:52:15.178 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/13' 2023-10-30T12:52:15.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/168/17' 2023-10-30T12:52:15.179 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/168' 2023-10-30T12:52:15.324 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:15.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:16.672 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/19' 2023-10-30T12:52:16.673 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/15' 2023-10-30T12:52:16.673 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/20' 2023-10-30T12:52:16.673 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/12' 2023-10-30T12:52:16.673 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/10' 2023-10-30T12:52:16.673 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/11' 2023-10-30T12:52:16.674 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/16' 2023-10-30T12:52:16.674 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/14' 2023-10-30T12:52:16.674 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/18' 2023-10-30T12:52:16.674 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/13' 2023-10-30T12:52:16.675 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/179/17' 2023-10-30T12:52:16.675 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/179' 2023-10-30T12:52:16.675 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/19' 2023-10-30T12:52:16.675 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/15' 2023-10-30T12:52:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/20' 2023-10-30T12:52:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/12' 2023-10-30T12:52:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/10' 2023-10-30T12:52:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/11' 2023-10-30T12:52:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/16' 2023-10-30T12:52:16.677 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/14' 2023-10-30T12:52:16.677 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/18' 2023-10-30T12:52:16.677 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/13' 2023-10-30T12:52:16.677 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/132/17' 2023-10-30T12:52:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/132' 2023-10-30T12:52:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/19' 2023-10-30T12:52:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/15' 2023-10-30T12:52:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/20' 2023-10-30T12:52:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/12' 2023-10-30T12:52:16.679 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/10' 2023-10-30T12:52:16.679 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/11' 2023-10-30T12:52:16.679 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/16' 2023-10-30T12:52:16.679 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/14' 2023-10-30T12:52:16.679 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/18' 2023-10-30T12:52:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/13' 2023-10-30T12:52:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/199/17' 2023-10-30T12:52:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/199' 2023-10-30T12:52:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/19' 2023-10-30T12:52:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/15' 2023-10-30T12:52:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/20' 2023-10-30T12:52:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/12' 2023-10-30T12:52:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/10' 2023-10-30T12:52:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/11' 2023-10-30T12:52:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/16' 2023-10-30T12:52:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/14' 2023-10-30T12:52:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/18' 2023-10-30T12:52:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/13' 2023-10-30T12:52:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/163/17' 2023-10-30T12:52:16.683 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/163' 2023-10-30T12:52:16.683 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/19' 2023-10-30T12:52:16.683 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/15' 2023-10-30T12:52:16.683 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/20' 2023-10-30T12:52:16.684 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/12' 2023-10-30T12:52:16.684 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/10' 2023-10-30T12:52:16.684 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/11' 2023-10-30T12:52:16.684 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/16' 2023-10-30T12:52:16.684 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/14' 2023-10-30T12:52:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/18' 2023-10-30T12:52:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/13' 2023-10-30T12:52:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/153/17' 2023-10-30T12:52:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/153' 2023-10-30T12:52:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/19' 2023-10-30T12:52:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/15' 2023-10-30T12:52:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/20' 2023-10-30T12:52:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/12' 2023-10-30T12:52:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/10' 2023-10-30T12:52:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/11' 2023-10-30T12:52:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/16' 2023-10-30T12:52:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/14' 2023-10-30T12:52:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/18' 2023-10-30T12:52:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/13' 2023-10-30T12:52:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/122/17' 2023-10-30T12:52:16.688 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/122' 2023-10-30T12:52:16.688 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/19' 2023-10-30T12:52:16.688 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/15' 2023-10-30T12:52:16.688 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/20' 2023-10-30T12:52:16.689 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/12' 2023-10-30T12:52:16.689 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/10' 2023-10-30T12:52:16.689 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/11' 2023-10-30T12:52:16.689 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/16' 2023-10-30T12:52:16.689 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/14' 2023-10-30T12:52:16.690 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/18' 2023-10-30T12:52:16.690 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/13' 2023-10-30T12:52:16.690 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/162/17' 2023-10-30T12:52:16.690 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/162' 2023-10-30T12:52:16.690 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/19' 2023-10-30T12:52:16.691 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/15' 2023-10-30T12:52:16.691 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/20' 2023-10-30T12:52:16.691 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/12' 2023-10-30T12:52:16.691 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/10' 2023-10-30T12:52:16.691 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/11' 2023-10-30T12:52:16.692 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/16' 2023-10-30T12:52:16.692 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/14' 2023-10-30T12:52:16.692 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/18' 2023-10-30T12:52:16.692 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/13' 2023-10-30T12:52:16.692 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/197/17' 2023-10-30T12:52:16.693 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/197' 2023-10-30T12:52:16.693 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/19' 2023-10-30T12:52:16.693 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/15' 2023-10-30T12:52:16.693 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/20' 2023-10-30T12:52:16.693 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/12' 2023-10-30T12:52:16.694 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/10' 2023-10-30T12:52:16.694 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/11' 2023-10-30T12:52:16.694 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/16' 2023-10-30T12:52:16.694 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/14' 2023-10-30T12:52:16.694 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/18' 2023-10-30T12:52:16.695 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/13' 2023-10-30T12:52:16.695 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/113/17' 2023-10-30T12:52:16.695 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/113' 2023-10-30T12:52:16.695 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/19' 2023-10-30T12:52:16.932 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/15' 2023-10-30T12:52:16.932 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/20' 2023-10-30T12:52:16.932 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/12' 2023-10-30T12:52:16.933 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/10' 2023-10-30T12:52:16.933 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/11' 2023-10-30T12:52:16.933 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/16' 2023-10-30T12:52:16.933 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/14' 2023-10-30T12:52:16.934 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/18' 2023-10-30T12:52:16.934 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/13' 2023-10-30T12:52:16.934 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/152/17' 2023-10-30T12:52:16.934 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/152' 2023-10-30T12:52:16.934 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/19' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/15' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/20' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/12' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/10' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/11' 2023-10-30T12:52:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/16' 2023-10-30T12:52:16.936 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/14' 2023-10-30T12:52:16.936 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/18' 2023-10-30T12:52:16.936 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/13' 2023-10-30T12:52:16.936 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/121/17' 2023-10-30T12:52:16.936 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/121' 2023-10-30T12:52:16.937 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/19' 2023-10-30T12:52:16.937 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/15' 2023-10-30T12:52:16.937 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/20' 2023-10-30T12:52:16.937 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/12' 2023-10-30T12:52:16.937 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/10' 2023-10-30T12:52:16.938 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/11' 2023-10-30T12:52:16.938 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/16' 2023-10-30T12:52:16.938 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/14' 2023-10-30T12:52:16.938 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/18' 2023-10-30T12:52:16.938 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/13' 2023-10-30T12:52:16.939 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/118/17' 2023-10-30T12:52:16.939 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/118' 2023-10-30T12:52:16.939 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/19' 2023-10-30T12:52:16.939 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/15' 2023-10-30T12:52:16.939 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/20' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/12' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/10' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/11' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/16' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/14' 2023-10-30T12:52:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/18' 2023-10-30T12:52:16.941 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/13' 2023-10-30T12:52:16.941 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/196/17' 2023-10-30T12:52:16.941 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/196' 2023-10-30T12:52:16.941 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/19' 2023-10-30T12:52:16.941 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/15' 2023-10-30T12:52:16.942 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/20' 2023-10-30T12:52:16.942 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/12' 2023-10-30T12:52:16.942 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/10' 2023-10-30T12:52:16.942 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/11' 2023-10-30T12:52:16.942 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/16' 2023-10-30T12:52:16.943 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/14' 2023-10-30T12:52:16.943 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/18' 2023-10-30T12:52:16.943 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/13' 2023-10-30T12:52:16.943 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/124/17' 2023-10-30T12:52:16.943 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/124' 2023-10-30T12:52:16.944 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/19' 2023-10-30T12:52:16.944 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/15' 2023-10-30T12:52:16.944 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/20' 2023-10-30T12:52:16.944 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/12' 2023-10-30T12:52:16.944 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/10' 2023-10-30T12:52:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/11' 2023-10-30T12:52:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/16' 2023-10-30T12:52:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/14' 2023-10-30T12:52:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/18' 2023-10-30T12:52:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/13' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/188/17' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/188' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/19' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/15' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/20' 2023-10-30T12:52:16.946 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/12' 2023-10-30T12:52:16.947 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/10' 2023-10-30T12:52:16.947 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/11' 2023-10-30T12:52:16.947 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/16' 2023-10-30T12:52:16.947 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/14' 2023-10-30T12:52:16.947 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/18' 2023-10-30T12:52:16.948 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/13' 2023-10-30T12:52:16.948 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/145/17' 2023-10-30T12:52:16.948 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/145' 2023-10-30T12:52:16.948 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/19' 2023-10-30T12:52:16.948 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/15' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/20' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/12' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/10' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/11' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/16' 2023-10-30T12:52:16.949 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/14' 2023-10-30T12:52:16.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/18' 2023-10-30T12:52:16.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/13' 2023-10-30T12:52:16.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/130/17' 2023-10-30T12:52:16.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/130' 2023-10-30T12:52:16.950 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/19' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/15' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/20' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/12' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/10' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/11' 2023-10-30T12:52:16.951 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/16' 2023-10-30T12:52:16.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/14' 2023-10-30T12:52:16.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/18' 2023-10-30T12:52:16.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/13' 2023-10-30T12:52:16.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/193/17' 2023-10-30T12:52:16.952 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/193' 2023-10-30T12:52:16.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/19' 2023-10-30T12:52:16.953 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/15' 2023-10-30T12:52:17.806 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/20' 2023-10-30T12:52:17.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/12' 2023-10-30T12:52:17.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/10' 2023-10-30T12:52:17.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/11' 2023-10-30T12:52:17.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/16' 2023-10-30T12:52:17.807 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/14' 2023-10-30T12:52:17.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/18' 2023-10-30T12:52:17.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/13' 2023-10-30T12:52:17.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/175/17' 2023-10-30T12:52:17.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/175' 2023-10-30T12:52:17.808 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/19' 2023-10-30T12:52:17.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/15' 2023-10-30T12:52:17.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/20' 2023-10-30T12:52:17.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/12' 2023-10-30T12:52:17.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/10' 2023-10-30T12:52:17.809 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/11' 2023-10-30T12:52:17.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/16' 2023-10-30T12:52:17.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/14' 2023-10-30T12:52:17.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/18' 2023-10-30T12:52:17.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/13' 2023-10-30T12:52:17.810 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/181/17' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/181' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/19' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/15' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/20' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/12' 2023-10-30T12:52:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/10' 2023-10-30T12:52:17.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/11' 2023-10-30T12:52:17.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/16' 2023-10-30T12:52:17.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/14' 2023-10-30T12:52:17.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/18' 2023-10-30T12:52:17.812 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/13' 2023-10-30T12:52:17.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/127/17' 2023-10-30T12:52:17.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/127' 2023-10-30T12:52:17.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/19' 2023-10-30T12:52:17.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/15' 2023-10-30T12:52:17.813 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/20' 2023-10-30T12:52:17.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/12' 2023-10-30T12:52:17.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/10' 2023-10-30T12:52:17.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/11' 2023-10-30T12:52:17.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/16' 2023-10-30T12:52:17.814 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/14' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/18' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/13' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/108/17' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/108' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/19' 2023-10-30T12:52:17.815 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/15' 2023-10-30T12:52:17.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/20' 2023-10-30T12:52:17.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/12' 2023-10-30T12:52:17.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/10' 2023-10-30T12:52:17.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/11' 2023-10-30T12:52:17.816 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/16' 2023-10-30T12:52:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/14' 2023-10-30T12:52:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/18' 2023-10-30T12:52:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/13' 2023-10-30T12:52:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/183/17' 2023-10-30T12:52:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/183' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/19' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/15' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/20' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/12' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/10' 2023-10-30T12:52:17.818 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/11' 2023-10-30T12:52:17.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/16' 2023-10-30T12:52:17.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/14' 2023-10-30T12:52:17.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/18' 2023-10-30T12:52:17.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/13' 2023-10-30T12:52:17.819 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/177/17' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/177' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/19' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/15' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/20' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/12' 2023-10-30T12:52:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/10' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/11' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/16' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/14' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/18' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/13' 2023-10-30T12:52:17.821 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/169/17' 2023-10-30T12:52:17.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/169' 2023-10-30T12:52:17.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/19' 2023-10-30T12:52:17.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/15' 2023-10-30T12:52:17.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/20' 2023-10-30T12:52:17.822 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/12' 2023-10-30T12:52:17.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/10' 2023-10-30T12:52:17.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/11' 2023-10-30T12:52:17.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/16' 2023-10-30T12:52:17.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/14' 2023-10-30T12:52:17.823 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/18' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/13' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/154/17' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/154' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/19' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/15' 2023-10-30T12:52:17.824 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/20' 2023-10-30T12:52:17.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/12' 2023-10-30T12:52:17.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/10' 2023-10-30T12:52:17.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/11' 2023-10-30T12:52:17.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/16' 2023-10-30T12:52:17.825 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/14' 2023-10-30T12:52:17.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/18' 2023-10-30T12:52:17.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/13' 2023-10-30T12:52:17.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/171/17' 2023-10-30T12:52:17.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/171' 2023-10-30T12:52:17.826 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/19' 2023-10-30T12:52:17.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/15' 2023-10-30T12:52:17.827 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/20' 2023-10-30T12:52:18.491 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/12' 2023-10-30T12:52:18.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/10' 2023-10-30T12:52:18.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/11' 2023-10-30T12:52:18.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/16' 2023-10-30T12:52:18.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/14' 2023-10-30T12:52:18.492 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/18' 2023-10-30T12:52:18.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/13' 2023-10-30T12:52:18.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/139/17' 2023-10-30T12:52:18.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/139' 2023-10-30T12:52:18.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/19' 2023-10-30T12:52:18.493 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/15' 2023-10-30T12:52:18.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/20' 2023-10-30T12:52:18.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/12' 2023-10-30T12:52:18.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/10' 2023-10-30T12:52:18.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/11' 2023-10-30T12:52:18.494 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/16' 2023-10-30T12:52:18.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/14' 2023-10-30T12:52:18.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/18' 2023-10-30T12:52:18.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/13' 2023-10-30T12:52:18.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/102/17' 2023-10-30T12:52:18.495 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/102' 2023-10-30T12:52:18.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/19' 2023-10-30T12:52:18.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/15' 2023-10-30T12:52:18.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/20' 2023-10-30T12:52:18.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/12' 2023-10-30T12:52:18.496 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/10' 2023-10-30T12:52:18.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/11' 2023-10-30T12:52:18.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/16' 2023-10-30T12:52:18.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/14' 2023-10-30T12:52:18.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/18' 2023-10-30T12:52:18.497 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/13' 2023-10-30T12:52:18.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed './copyofsnap1/test1/129/17' 2023-10-30T12:52:18.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1/129' 2023-10-30T12:52:18.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1/test1' 2023-10-30T12:52:18.498 INFO:tasks.workunit.client.0.smithi008.stdout:removed directory './copyofsnap1' 2023-10-30T12:52:18.498 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:18.500 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:18.500 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:18.771 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2023-10-30T12:52:18.771 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2023-10-30T12:52:18.789 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-10-30T12:52:18.789 INFO:tasks.workunit.client.0.smithi008.stderr:+ touch foo 2023-10-30T12:52:18.791 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod +x foo 2023-10-30T12:52:18.792 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir .snap/s 2023-10-30T12:52:18.806 INFO:tasks.workunit.client.0.smithi008.stderr:+ find .snap/s/foo -executable 2023-10-30T12:52:18.807 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep foo 2023-10-30T12:52:18.810 INFO:tasks.workunit.client.0.smithi008.stdout:.snap/s/foo 2023-10-30T12:52:18.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir .snap/s 2023-10-30T12:52:18.820 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm foo 2023-10-30T12:52:18.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:52:18.825 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:18.826 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:18.826 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:20.013 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2023-10-30T12:52:20.013 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2023-10-30T12:52:24.171 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-10-30T12:52:24.171 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir foo 2023-10-30T12:52:24.174 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo x 2023-10-30T12:52:24.175 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:25.737 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir foo/.snap/ss 2023-10-30T12:52:25.750 INFO:tasks.workunit.client.0.smithi008.stderr:+ ln foo/x foo/xx 2023-10-30T12:52:25.752 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat foo/.snap/ss/x 2023-10-30T12:52:25.759 INFO:tasks.workunit.client.0.smithi008.stdout:x 2023-10-30T12:52:25.760 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir foo/.snap/ss 2023-10-30T12:52:26.647 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo a 2023-10-30T12:52:26.647 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo b 2023-10-30T12:52:26.647 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir foo/.snap/s 2023-10-30T12:52:26.660 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat foo/.snap/s/a 2023-10-30T12:52:26.670 INFO:tasks.workunit.client.0.smithi008.stderr:+ r=a 2023-10-30T12:52:26.671 INFO:tasks.workunit.client.0.smithi008.stderr:+ [ -z a ] 2023-10-30T12:52:26.671 INFO:tasks.workunit.client.0.smithi008.stderr:+ ln foo/b foo/b2 2023-10-30T12:52:26.672 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat foo/.snap/s/b 2023-10-30T12:52:26.683 INFO:tasks.workunit.client.0.smithi008.stdout:b 2023-10-30T12:52:26.684 INFO:tasks.workunit.client.0.smithi008.stdout:this used to hang: 2023-10-30T12:52:26.684 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo this used to hang: 2023-10-30T12:52:26.684 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo more 2023-10-30T12:52:26.685 INFO:tasks.workunit.client.0.smithi008.stdout:oh, it didn't hang! good job. 2023-10-30T12:52:26.685 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo oh, it didn't hang! good job. 2023-10-30T12:52:26.685 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat foo/b 2023-10-30T12:52:26.686 INFO:tasks.workunit.client.0.smithi008.stdout:b 2023-10-30T12:52:26.686 INFO:tasks.workunit.client.0.smithi008.stdout:more 2023-10-30T12:52:26.686 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir foo/.snap/s 2023-10-30T12:52:26.693 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm -r foo 2023-10-30T12:52:26.709 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:52:26.711 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:26.711 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:26.711 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:26.989 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2023-10-30T12:52:26.990 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2023-10-30T12:52:27.007 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-10-30T12:52:27.007 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:27.009 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir .snap/bar 2023-10-30T12:52:27.024 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/foo 2023-10-30T12:52:27.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/foo 2023-10-30T12:52:27.041 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:27.044 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/bar/a 2023-10-30T12:52:27.047 INFO:tasks.workunit.client.0.smithi008.stdout: File: .snap/bar/a 2023-10-30T12:52:27.047 INFO:tasks.workunit.client.0.smithi008.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-30T12:52:27.047 INFO:tasks.workunit.client.0.smithi008.stdout:Device: fah/250d Inode: 1099511632710 Links: 2 2023-10-30T12:52:27.047 INFO:tasks.workunit.client.0.smithi008.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2023-10-30T12:52:27.048 INFO:tasks.workunit.client.0.smithi008.stdout:Access: 2023-10-30 12:52:27.005267588 +0000 2023-10-30T12:52:27.048 INFO:tasks.workunit.client.0.smithi008.stdout:Modify: 2023-10-30 12:52:27.005267588 +0000 2023-10-30T12:52:27.048 INFO:tasks.workunit.client.0.smithi008.stdout:Change: 2023-10-30 12:52:27.005267588 +0000 2023-10-30T12:52:27.048 INFO:tasks.workunit.client.0.smithi008.stdout: Birth: 2023-10-30 12:52:27.005267588 +0000 2023-10-30T12:52:27.048 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir .snap/bar 2023-10-30T12:52:27.059 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:52:27.060 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:27.060 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:27.060 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:27.257 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2023-10-30T12:52:27.257 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2023-10-30T12:52:27.275 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-10-30T12:52:27.275 INFO:tasks.workunit.client.0.smithi008.stderr:+ seq 1 20 2023-10-30T12:52:27.276 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:27.278 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:27.279 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:27.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:27.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:27.303 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:27.463 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:27.478 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:27.478 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:27.479 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:27.479 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:27.479 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:27.479 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:27.479 INFO:tasks.workunit.client.0.smithi008.stdout: 48523 ? 00:00:00 ps 2023-10-30T12:52:27.480 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:27.495 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:27.497 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:27.505 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:27.507 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:27.508 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:27.520 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:27.528 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:27.529 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:27.586 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:27.601 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:27.601 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:27.601 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:27.602 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:27.602 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:27.602 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:27.602 INFO:tasks.workunit.client.0.smithi008.stdout: 48533 ? 00:00:00 ps 2023-10-30T12:52:27.603 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:27.618 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:27.622 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:27.630 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:27.633 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:27.634 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:27.646 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:27.656 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:27.657 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:27.721 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:27.736 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:27.736 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:27.736 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:27.737 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:27.737 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:27.737 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:27.737 INFO:tasks.workunit.client.0.smithi008.stdout: 48543 ? 00:00:00 ps 2023-10-30T12:52:27.738 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:27.753 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:27.756 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:27.763 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:27.766 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:27.767 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:27.779 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:27.788 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:27.789 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:28.103 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:28.118 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:28.118 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:28.118 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:28.119 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:28.119 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:28.119 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:28.119 INFO:tasks.workunit.client.0.smithi008.stdout: 48553 ? 00:00:00 ps 2023-10-30T12:52:28.120 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:28.592 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:28.597 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:29.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:29.144 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:29.145 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:29.162 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:29.194 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:29.195 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:29.380 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:29.395 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:29.395 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:29.396 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:29.396 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:29.396 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:29.396 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:29.396 INFO:tasks.workunit.client.0.smithi008.stdout: 48564 ? 00:00:00 ps 2023-10-30T12:52:29.397 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:29.422 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:29.425 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:29.434 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:29.437 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:29.438 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:29.452 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:29.460 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:29.461 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:29.596 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:29.612 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:29.612 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:29.612 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:29.612 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:29.613 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:29.613 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:29.613 INFO:tasks.workunit.client.0.smithi008.stdout: 48574 ? 00:00:00 ps 2023-10-30T12:52:29.613 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:29.629 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:29.632 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:29.639 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:29.642 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:29.643 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:29.656 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:29.664 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:29.665 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:29.755 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:29.770 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:29.770 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:29.770 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:29.771 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:29.771 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:29.771 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:29.771 INFO:tasks.workunit.client.0.smithi008.stdout: 48584 ? 00:00:00 ps 2023-10-30T12:52:29.772 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:29.788 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:29.790 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:29.798 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:29.801 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:29.802 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:29.817 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:30.485 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:30.487 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:30.572 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:30.587 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:30.587 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:30.587 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:30.588 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:30.588 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:30.588 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:30.588 INFO:tasks.workunit.client.0.smithi008.stdout: 48594 ? 00:00:00 ps 2023-10-30T12:52:30.589 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:30.605 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:30.608 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:30.865 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:30.867 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:30.869 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:30.882 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:30.891 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:30.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:30.981 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:30.997 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:30.997 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:30.997 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:30.997 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:30.998 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:30.998 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:30.998 INFO:tasks.workunit.client.0.smithi008.stdout: 48604 ? 00:00:00 ps 2023-10-30T12:52:30.999 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:31.020 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:31.022 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:31.030 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:31.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:31.034 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:31.047 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:31.056 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:31.057 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:31.131 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:31.146 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:31.147 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:31.147 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:31.147 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:31.147 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:31.147 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:31.148 INFO:tasks.workunit.client.0.smithi008.stdout: 48614 ? 00:00:00 ps 2023-10-30T12:52:31.148 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:31.165 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:31.168 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:31.177 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:31.181 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:31.183 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:31.278 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:31.287 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:31.289 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:31.424 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:31.443 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:31.443 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:31.443 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:31.443 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:31.444 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:31.444 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:31.444 INFO:tasks.workunit.client.0.smithi008.stdout: 48624 ? 00:00:00 ps 2023-10-30T12:52:31.444 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:31.462 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:31.464 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:31.473 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:31.475 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:31.476 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:31.490 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:31.499 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:31.500 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:31.607 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:31.622 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:31.622 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:31.622 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:31.622 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:31.623 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:31.623 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:31.623 INFO:tasks.workunit.client.0.smithi008.stdout: 48634 ? 00:00:00 ps 2023-10-30T12:52:31.623 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:31.640 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:31.642 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:31.649 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:31.652 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:31.653 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:31.665 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:31.674 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:31.675 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:31.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:31.887 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:31.887 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:31.887 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:31.887 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:31.888 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:31.888 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:31.888 INFO:tasks.workunit.client.0.smithi008.stdout: 48644 ? 00:00:00 ps 2023-10-30T12:52:31.888 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:31.904 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:31.906 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:31.913 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:31.916 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:31.916 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:31.929 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:31.938 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:31.939 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.016 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.031 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.031 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.031 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.031 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.032 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.032 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.032 INFO:tasks.workunit.client.0.smithi008.stdout: 48654 ? 00:00:00 ps 2023-10-30T12:52:32.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.048 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.051 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.057 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.060 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.061 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.074 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.083 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.084 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.132 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.148 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.148 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.148 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.148 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.149 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.149 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.149 INFO:tasks.workunit.client.0.smithi008.stdout: 48664 ? 00:00:00 ps 2023-10-30T12:52:32.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.166 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.168 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.175 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.179 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.181 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.285 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.286 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.391 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.406 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.407 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.407 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.407 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.407 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.408 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.408 INFO:tasks.workunit.client.0.smithi008.stdout: 48674 ? 00:00:00 ps 2023-10-30T12:52:32.408 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.423 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.425 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.435 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.439 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.441 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.459 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.468 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.469 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.559 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.573 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.574 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.574 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.574 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.575 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.575 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.575 INFO:tasks.workunit.client.0.smithi008.stdout: 48684 ? 00:00:00 ps 2023-10-30T12:52:32.576 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.590 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.593 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.600 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.603 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.603 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.617 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.626 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.627 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.698 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.699 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.699 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.699 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.699 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.699 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.700 INFO:tasks.workunit.client.0.smithi008.stdout: 48694 ? 00:00:00 ps 2023-10-30T12:52:32.700 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.716 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.718 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.725 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.728 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.729 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.742 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.752 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.833 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.833 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.833 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.833 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.834 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.834 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.834 INFO:tasks.workunit.client.0.smithi008.stdout: 48704 ? 00:00:00 ps 2023-10-30T12:52:32.835 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.849 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.852 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:32.859 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a 2023-10-30T12:52:32.861 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/one 2023-10-30T12:52:32.862 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat 2023-10-30T12:52:32.876 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir a/.snap/two 2023-10-30T12:52:32.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 777 a/foo 2023-10-30T12:52:32.886 INFO:tasks.workunit.client.0.smithi008.stderr:+ sync 2023-10-30T12:52:32.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ ps 2023-10-30T12:52:32.966 INFO:tasks.workunit.client.0.smithi008.stdout: PID TTY TIME CMD 2023-10-30T12:52:32.966 INFO:tasks.workunit.client.0.smithi008.stdout: 1038 ? 00:00:00 systemd 2023-10-30T12:52:32.967 INFO:tasks.workunit.client.0.smithi008.stdout: 1039 ? 00:00:00 (sd-pam) 2023-10-30T12:52:32.967 INFO:tasks.workunit.client.0.smithi008.stdout: 27924 ? 00:00:01 sshd 2023-10-30T12:52:32.967 INFO:tasks.workunit.client.0.smithi008.stdout: 48512 ? 00:00:00 timeout 2023-10-30T12:52:32.967 INFO:tasks.workunit.client.0.smithi008.stdout: 48515 ? 00:00:00 snaptest-double 2023-10-30T12:52:32.968 INFO:tasks.workunit.client.0.smithi008.stdout: 48714 ? 00:00:00 ps 2023-10-30T12:52:32.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-30T12:52:32.991 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm a/foo 2023-10-30T12:52:32.993 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir a 2023-10-30T12:52:33.001 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:52:33.002 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:33.003 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:33.003 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:33.088 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2023-10-30T12:52:33.088 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2023-10-30T12:52:34.169 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir .snap/foo 2023-10-30T12:52:34.180 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo We want ENOENT, not ESTALE, here. 2023-10-30T12:52:34.181 INFO:tasks.workunit.client.0.smithi008.stdout:We want ENOENT, not ESTALE, here. 2023-10-30T12:52:34.181 INFO:tasks.workunit.client.0.smithi008.stderr:+ seq 1 100 2023-10-30T12:52:34.182 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/1 2023-10-30T12:52:34.185 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.188 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2023-10-30T12:52:34.188 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/2 2023-10-30T12:52:34.189 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.800 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2023-10-30T12:52:34.801 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/3 2023-10-30T12:52:34.801 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.802 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2023-10-30T12:52:34.805 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/4 2023-10-30T12:52:34.806 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.806 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2023-10-30T12:52:34.807 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/5 2023-10-30T12:52:34.807 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.807 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2023-10-30T12:52:34.808 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/6 2023-10-30T12:52:34.808 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.809 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2023-10-30T12:52:34.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/7 2023-10-30T12:52:34.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.812 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2023-10-30T12:52:34.812 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/8 2023-10-30T12:52:34.813 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.814 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2023-10-30T12:52:34.815 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/9 2023-10-30T12:52:34.815 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.817 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2023-10-30T12:52:34.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/10 2023-10-30T12:52:34.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.819 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2023-10-30T12:52:34.820 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/11 2023-10-30T12:52:34.820 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.821 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2023-10-30T12:52:34.822 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/12 2023-10-30T12:52:34.822 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.824 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2023-10-30T12:52:34.824 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/13 2023-10-30T12:52:34.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.826 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2023-10-30T12:52:34.827 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/14 2023-10-30T12:52:34.828 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.830 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2023-10-30T12:52:34.831 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/15 2023-10-30T12:52:34.832 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.834 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2023-10-30T12:52:34.835 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/16 2023-10-30T12:52:34.836 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.838 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2023-10-30T12:52:34.840 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.840 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/17 2023-10-30T12:52:34.844 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2023-10-30T12:52:34.845 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/18 2023-10-30T12:52:34.846 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.847 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2023-10-30T12:52:34.848 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/19 2023-10-30T12:52:34.848 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.850 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2023-10-30T12:52:34.851 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/20 2023-10-30T12:52:34.851 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.852 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2023-10-30T12:52:34.853 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/21 2023-10-30T12:52:34.853 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.855 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2023-10-30T12:52:34.855 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/22 2023-10-30T12:52:34.856 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.857 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2023-10-30T12:52:34.858 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/23 2023-10-30T12:52:34.858 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.859 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2023-10-30T12:52:34.860 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/24 2023-10-30T12:52:34.860 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.861 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2023-10-30T12:52:34.862 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/25 2023-10-30T12:52:34.862 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.864 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2023-10-30T12:52:34.864 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/26 2023-10-30T12:52:34.865 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.866 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2023-10-30T12:52:34.867 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/27 2023-10-30T12:52:34.867 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.868 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2023-10-30T12:52:34.869 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/28 2023-10-30T12:52:34.869 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.871 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2023-10-30T12:52:34.871 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/29 2023-10-30T12:52:34.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.873 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2023-10-30T12:52:34.874 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/30 2023-10-30T12:52:34.874 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.875 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2023-10-30T12:52:34.876 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/31 2023-10-30T12:52:34.876 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.877 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2023-10-30T12:52:34.878 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/32 2023-10-30T12:52:34.879 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.880 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2023-10-30T12:52:34.880 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/33 2023-10-30T12:52:34.881 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.882 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2023-10-30T12:52:34.883 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/34 2023-10-30T12:52:34.883 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.884 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2023-10-30T12:52:34.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/35 2023-10-30T12:52:34.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.887 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2023-10-30T12:52:34.887 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/36 2023-10-30T12:52:34.888 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.889 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2023-10-30T12:52:34.890 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/37 2023-10-30T12:52:34.890 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.891 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2023-10-30T12:52:34.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/38 2023-10-30T12:52:34.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.893 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2023-10-30T12:52:34.894 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/39 2023-10-30T12:52:34.895 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.896 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2023-10-30T12:52:34.897 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/40 2023-10-30T12:52:34.897 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.898 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2023-10-30T12:52:34.899 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/41 2023-10-30T12:52:34.899 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.900 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2023-10-30T12:52:34.901 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/42 2023-10-30T12:52:34.902 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.903 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2023-10-30T12:52:34.904 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/43 2023-10-30T12:52:34.904 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.905 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2023-10-30T12:52:34.906 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/44 2023-10-30T12:52:34.906 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.907 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2023-10-30T12:52:34.908 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/45 2023-10-30T12:52:34.909 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.910 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2023-10-30T12:52:34.910 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/46 2023-10-30T12:52:34.911 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.912 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2023-10-30T12:52:34.913 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/47 2023-10-30T12:52:34.913 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.914 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2023-10-30T12:52:34.915 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/48 2023-10-30T12:52:34.916 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.917 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2023-10-30T12:52:34.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/49 2023-10-30T12:52:34.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.919 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2023-10-30T12:52:34.920 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/50 2023-10-30T12:52:34.920 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.921 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2023-10-30T12:52:34.922 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/51 2023-10-30T12:52:34.922 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.924 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2023-10-30T12:52:34.925 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/52 2023-10-30T12:52:34.925 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2023-10-30T12:52:34.927 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/53 2023-10-30T12:52:34.927 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.928 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2023-10-30T12:52:34.929 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/54 2023-10-30T12:52:34.929 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.931 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2023-10-30T12:52:34.932 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/55 2023-10-30T12:52:34.932 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.933 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2023-10-30T12:52:34.934 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/56 2023-10-30T12:52:34.934 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.935 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2023-10-30T12:52:34.936 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/57 2023-10-30T12:52:34.937 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.938 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2023-10-30T12:52:34.939 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/58 2023-10-30T12:52:34.939 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.940 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2023-10-30T12:52:34.941 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/59 2023-10-30T12:52:34.941 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.943 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2023-10-30T12:52:34.944 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/60 2023-10-30T12:52:34.944 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.945 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2023-10-30T12:52:34.946 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/61 2023-10-30T12:52:34.946 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.948 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2023-10-30T12:52:34.948 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/62 2023-10-30T12:52:34.949 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.950 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2023-10-30T12:52:34.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/63 2023-10-30T12:52:34.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.952 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2023-10-30T12:52:34.953 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/64 2023-10-30T12:52:34.953 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.955 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2023-10-30T12:52:34.956 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/65 2023-10-30T12:52:34.956 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.957 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2023-10-30T12:52:34.958 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/66 2023-10-30T12:52:34.958 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.960 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2023-10-30T12:52:34.960 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/67 2023-10-30T12:52:34.961 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.962 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2023-10-30T12:52:34.963 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/68 2023-10-30T12:52:34.963 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.964 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2023-10-30T12:52:34.965 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/69 2023-10-30T12:52:34.966 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:34.967 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2023-10-30T12:52:34.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/70 2023-10-30T12:52:34.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.219 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2023-10-30T12:52:35.220 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/71 2023-10-30T12:52:35.221 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.222 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2023-10-30T12:52:35.223 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/72 2023-10-30T12:52:35.223 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.224 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2023-10-30T12:52:35.225 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/73 2023-10-30T12:52:35.225 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.227 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2023-10-30T12:52:35.227 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/74 2023-10-30T12:52:35.228 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.229 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2023-10-30T12:52:35.230 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/75 2023-10-30T12:52:35.230 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.232 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2023-10-30T12:52:35.232 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/76 2023-10-30T12:52:35.233 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.234 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2023-10-30T12:52:35.235 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/77 2023-10-30T12:52:35.235 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.236 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2023-10-30T12:52:35.237 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/78 2023-10-30T12:52:35.237 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.238 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2023-10-30T12:52:35.239 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/79 2023-10-30T12:52:35.239 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.241 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2023-10-30T12:52:35.241 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/80 2023-10-30T12:52:35.242 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.243 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2023-10-30T12:52:35.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/81 2023-10-30T12:52:35.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.245 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2023-10-30T12:52:35.246 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/82 2023-10-30T12:52:35.247 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.248 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2023-10-30T12:52:35.248 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/83 2023-10-30T12:52:35.249 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.250 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2023-10-30T12:52:35.251 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/84 2023-10-30T12:52:35.251 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.252 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2023-10-30T12:52:35.253 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/85 2023-10-30T12:52:35.253 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.255 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2023-10-30T12:52:35.255 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/86 2023-10-30T12:52:35.256 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.257 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2023-10-30T12:52:35.258 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/87 2023-10-30T12:52:35.258 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.259 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2023-10-30T12:52:35.260 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/88 2023-10-30T12:52:35.261 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.268 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2023-10-30T12:52:35.268 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/89 2023-10-30T12:52:35.269 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.270 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2023-10-30T12:52:35.271 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/90 2023-10-30T12:52:35.271 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.272 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2023-10-30T12:52:35.273 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/91 2023-10-30T12:52:35.273 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.275 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2023-10-30T12:52:35.276 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/92 2023-10-30T12:52:35.276 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.277 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2023-10-30T12:52:35.278 INFO:tasks.workunit.client.0.smithi008.stderr:+ + stat .snap/foo/93 2023-10-30T12:52:35.278 INFO:tasks.workunit.client.0.smithi008.stderr:grep No such file 2023-10-30T12:52:35.280 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2023-10-30T12:52:35.281 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/94 2023-10-30T12:52:35.281 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.282 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2023-10-30T12:52:35.283 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/95 2023-10-30T12:52:35.283 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.284 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2023-10-30T12:52:35.285 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/96 2023-10-30T12:52:35.285 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.287 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2023-10-30T12:52:35.288 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/97 2023-10-30T12:52:35.288 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.289 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2023-10-30T12:52:35.290 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/98 2023-10-30T12:52:35.290 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.291 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2023-10-30T12:52:35.292 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/99 2023-10-30T12:52:35.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.294 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2023-10-30T12:52:35.295 INFO:tasks.workunit.client.0.smithi008.stderr:+ stat .snap/foo/100 2023-10-30T12:52:35.295 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep No such file 2023-10-30T12:52:35.296 INFO:tasks.workunit.client.0.smithi008.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2023-10-30T12:52:35.296 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir .snap/foo 2023-10-30T12:52:35.312 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-10-30T12:52:35.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-10-30T12:52:35.313 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-30T12:52:35.314 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-30T12:52:35.916 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2023-10-30T12:52:35.916 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2023-10-30T12:52:35.934 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-10-30T12:52:35.934 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo git config --global http.sslVerify false 2023-10-30T12:52:35.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo git config --global http.postBuffer 1048576000 2023-10-30T12:52:35.965 INFO:tasks.workunit.client.0.smithi008.stderr:+ retried=false 2023-10-30T12:52:35.966 INFO:tasks.workunit.client.0.smithi008.stderr:+ trap -- retry EXIT 2023-10-30T12:52:35.966 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm -rf ceph 2023-10-30T12:52:35.966 INFO:tasks.workunit.client.0.smithi008.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2023-10-30T12:52:35.971 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into 'ceph'... 2023-10-30T12:52:45.386 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:45.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:52:47.520 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 Processor -- start 2023-10-30T12:52:47.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 -- start start 2023-10-30T12:52:47.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd3c0cb7d0 0x7efd3c0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efd3c0c3e40 0x7efd3c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c0c2310 con 0x7efd3c150830 2023-10-30T12:52:47.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c06bc10 con 0x7efd3c0c3e40 2023-10-30T12:52:47.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd4266b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c156710 con 0x7efd3c0cb7d0 2023-10-30T12:52:47.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd3bfff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7efd3c150830 0x7efd3c06b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55916/0 (socket says 172.21.15.8:55916) 2023-10-30T12:52:47.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd3bfff640 1 -- 172.21.15.8:0/695544421 learned_addr learned my addr 172.21.15.8:0/695544421 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:52:47.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd40c08640 1 --2- 172.21.15.8:0/695544421 >> v2:172.21.15.8:3300/0 conn(0x7efd3c0c3e40 0x7efd3c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd3b7fe640 1 --2- 172.21.15.8:0/695544421 >> v2:172.21.15.46:3300/0 conn(0x7efd3c0cb7d0 0x7efd3c0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd40c08640 1 -- 172.21.15.8:0/695544421 >> v2:172.21.15.46:3300/0 conn(0x7efd3c0cb7d0 msgr2=0x7efd3c0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:47.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd40c08640 1 --2- 172.21.15.8:0/695544421 >> v2:172.21.15.46:3300/0 conn(0x7efd3c0cb7d0 0x7efd3c0c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd40c08640 1 -- 172.21.15.8:0/695544421 >> v1:172.21.15.8:6789/0 conn(0x7efd3c150830 legacy=0x7efd3c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd40c08640 1 -- 172.21.15.8:0/695544421 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0c4360 con 0x7efd3c0c3e40 2023-10-30T12:52:47.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.673+0000 7efd3affd640 1 -- 172.21.15.8:0/695544421 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965328521 0 0) 0x7efd3c0c2310 con 0x7efd3c150830 2023-10-30T12:52:47.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd40c08640 1 --2- 172.21.15.8:0/695544421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c070520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd3c06f9f0 tx=0x7efd3401f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99516d497c789c8e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:47.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd3affd640 1 -- 172.21.15.8:0/695544421 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340b0440 con 0x7efd3c0c3e40 2023-10-30T12:52:47.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd3affd640 1 -- 172.21.15.8:0/695544421 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340b05a0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd3affd640 1 -- 172.21.15.8:0/695544421 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340b0930 con 0x7efd3c0c3e40 2023-10-30T12:52:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 -- 172.21.15.8:0/695544421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 msgr2=0x7efd3c070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 --2- 172.21.15.8:0/695544421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c070520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd3c06f9f0 tx=0x7efd3401f0f0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 -- 172.21.15.8:0/695544421 shutdown_connections 2023-10-30T12:52:47.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 --2- 172.21.15.8:0/695544421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c070520 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 --2- 172.21.15.8:0/695544421 >> v2:172.21.15.46:3300/0 conn(0x7efd3c0cb7d0 0x7efd3c0c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 -- 172.21.15.8:0/695544421 >> 172.21.15.8:0/695544421 conn(0x7efd3c0cac70 msgr2=0x7efd3c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 -- 172.21.15.8:0/695544421 shutdown_connections 2023-10-30T12:52:47.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.677+0000 7efd4266b640 1 -- 172.21.15.8:0/695544421 wait complete. 2023-10-30T12:52:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 Processor -- start 2023-10-30T12:52:47.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- start start 2023-10-30T12:52:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efd3c0cb7d0 0x7efd3c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 0x7efd3c1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c152a20 con 0x7efd3c0c3e40 2023-10-30T12:52:47.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c1f4060 con 0x7efd3c150830 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c152050 con 0x7efd3c0cb7d0 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45966/0 (socket says 172.21.15.8:45966) 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 -- 172.21.15.8:0/1711610842 learned_addr learned my addr 172.21.15.8:0/1711610842 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd40c08640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 0x7efd3c1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3b7fe640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efd3c0cb7d0 0x7efd3c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 -- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efd3c0cb7d0 msgr2=0x7efd3c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efd3c0cb7d0 0x7efd3c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 -- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 msgr2=0x7efd3c1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 0x7efd3c1f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 -- 172.21.15.8:0/1711610842 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009a70 con 0x7efd3c0c3e40 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd40c08640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 0x7efd3c1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd3bfff640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c580 tx=0x7efd2c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c005cf0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c005e70 con 0x7efd3c0c3e40 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c1516f0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c007310 con 0x7efd3c0c3e40 2023-10-30T12:52:47.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.681+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c151410 con 0x7efd3c0c3e40 2023-10-30T12:52:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c04e170 con 0x7efd3c0c3e40 2023-10-30T12:52:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd2c007470 con 0x7efd3c0c3e40 2023-10-30T12:52:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd38ff9640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 0x7efd28068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7efd2c0836a0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd3b7fe640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 0x7efd28068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.685+0000 7efd3b7fe640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 0x7efd28068990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd3c14d0b0 tx=0x7efd30008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:47.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.689+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd2c0508e0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.881+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd3c0c71a0 con 0x7efd3c0c3e40 2023-10-30T12:52:47.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.881+0000 7efd38ff9640 1 -- 172.21.15.8:0/1711610842 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7efd2c053f00 con 0x7efd3c0c3e40 2023-10-30T12:52:47.885 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:47.886 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:47.886 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:47.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 msgr2=0x7efd28068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 0x7efd28068990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd3c14d0b0 tx=0x7efd30008040 comp rx=0 tx=0).stop 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 msgr2=0x7efd3c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c580 tx=0x7efd2c00ca40 comp rx=0 tx=0).stop 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 shutdown_connections 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efd280664d0 0x7efd28068990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd3c150830 0x7efd3c1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efd3c0cb7d0 0x7efd3c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 --2- 172.21.15.8:0/1711610842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0c3e40 0x7efd3c1577c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:47.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 >> 172.21.15.8:0/1711610842 conn(0x7efd3c0cac70 msgr2=0x7efd3c14ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:47.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 shutdown_connections 2023-10-30T12:52:47.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:47.885+0000 7efd4266b640 1 -- 172.21.15.8:0/1711610842 wait complete. 2023-10-30T12:52:47.906 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2023-10-30T12:52:47.907 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-30T12:52:47.907 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:52:47.924 INFO:tasks.ceph.mds.a:Stopped 2023-10-30T12:52:47.924 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:48.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.105+0000 7f2b3fdd3640 1 Processor -- start 2023-10-30T12:52:48.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.105+0000 7f2b3fdd3640 1 -- start start 2023-10-30T12:52:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3fdd3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b3806bb40 0x7f2b38157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3fdd3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b3806c420 0x7f2b38157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3fdd3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b38071e00 con 0x7f2b3806b760 2023-10-30T12:52:48.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3fdd3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b38070020 con 0x7f2b3806bb40 2023-10-30T12:52:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3fdd3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38070f10 con 0x7f2b3806c420 2023-10-30T12:52:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3db6f640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2b3806b760 0x7f2b3815e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55932/0 (socket says 172.21.15.8:55932) 2023-10-30T12:52:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3db6f640 1 -- 172.21.15.8:0/187767889 learned_addr learned my addr 172.21.15.8:0/187767889 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:52:48.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3d36e640 1 --2- 172.21.15.8:0/187767889 >> v2:172.21.15.46:3300/0 conn(0x7f2b3806bb40 0x7f2b38157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b3806c420 0x7f2b38157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538571420 0 0) 0x7f2b38071e00 con 0x7f2b3806b760 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b24003530 con 0x7f2b3806b760 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1426204925 0 0) 0x7f2b24003530 con 0x7f2b3806b760 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b3806c420 msgr2=0x7f2b38157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:48.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b3806c420 0x7f2b38157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 >> v2:172.21.15.46:3300/0 conn(0x7f2b3806bb40 msgr2=0x7f2b38157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 --2- 172.21.15.8:0/187767889 >> v2:172.21.15.46:3300/0 conn(0x7f2b3806bb40 0x7f2b38157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380711f0 con 0x7f2b3806b760 2023-10-30T12:52:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f2b34001680 con 0x7f2b3806b760 2023-10-30T12:52:48.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 >> v1:172.21.15.8:6789/0 conn(0x7f2b3806b760 legacy=0x7f2b3815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b24003980 0x7f2b24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004370 0x7f2b24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b24004f70 0x7f2b2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38070020 con 0x7f2b24004f70 2023-10-30T12:52:48.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38070f10 con 0x7f2b24004370 2023-10-30T12:52:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2400bf00 con 0x7f2b24003980 2023-10-30T12:52:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b3806c420 0x7f2b38157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b24004f70 0x7f2b2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3db6f640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b24003980 0x7f2b24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3d36e640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004370 0x7f2b24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 -- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b24003980 msgr2=0x7f2b24003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b24003980 0x7f2b24003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 -- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004370 msgr2=0x7f2b24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004370 0x7f2b24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.109+0000 7f2b3e370640 1 -- 172.21.15.8:0/187767889 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24003530 con 0x7f2b24004f70 2023-10-30T12:52:48.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3e370640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b24004f70 0x7f2b2400b990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b3806c800 tx=0x7f2b3001ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bf8752299ebd93 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:48.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a5be0 con 0x7f2b24004f70 2023-10-30T12:52:48.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300a5d40 con 0x7f2b24004f70 2023-10-30T12:52:48.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3cb6d640 1 -- 172.21.15.8:0/187767889 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30003bc0 con 0x7f2b24004f70 2023-10-30T12:52:48.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/187767889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b24004f70 msgr2=0x7f2b2400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/187767889 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b24004f70 0x7f2b2400b990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b3806c800 tx=0x7f2b3001ccf0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3e370640 1 -- 172.21.15.8:0/187767889 reap_dead start 2023-10-30T12:52:48.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/187767889 shutdown_connections 2023-10-30T12:52:48.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/187767889 >> 172.21.15.8:0/187767889 conn(0x7f2b380cac70 msgr2=0x7f2b380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:48.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/187767889 shutdown_connections 2023-10-30T12:52:48.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.113+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/187767889 wait complete. 2023-10-30T12:52:48.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 Processor -- start 2023-10-30T12:52:48.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- start start 2023-10-30T12:52:48.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 0x7f2b380c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 0x7f2b380c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380ca4d0 con 0x7f2b380c3800 2023-10-30T12:52:48.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38161180 con 0x7f2b24004f70 2023-10-30T12:52:48.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380c3c70 con 0x7f2b38161dd0 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45978/0 (socket says 172.21.15.8:45978) 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 -- 172.21.15.8:0/3767793556 learned_addr learned my addr 172.21.15.8:0/3767793556 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3db6f640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 0x7f2b380c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3d36e640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 0x7f2b380c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 -- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 msgr2=0x7f2b380c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 0x7f2b380c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 -- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 msgr2=0x7f2b380c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 0x7f2b380c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 -- 172.21.15.8:0/3767793556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380c9c30 con 0x7f2b380c3800 2023-10-30T12:52:48.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3db6f640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 0x7f2b380c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:48.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3d36e640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 0x7f2b380c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:48.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3e370640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b3001c760 tx=0x7f2b24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:48.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30004440 con 0x7f2b380c3800 2023-10-30T12:52:48.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300045a0 con 0x7f2b380c3800 2023-10-30T12:52:48.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30003e60 con 0x7f2b380c3800 2023-10-30T12:52:48.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b380c9f10 con 0x7f2b380c3800 2023-10-30T12:52:48.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b380c2ac0 con 0x7f2b380c3800 2023-10-30T12:52:48.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.117+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b380c2920 con 0x7f2b380c3800 2023-10-30T12:52:48.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.121+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b300b0460 con 0x7f2b380c3800 2023-10-30T12:52:48.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.121+0000 7f2b2e7fc640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 0x7f2b1c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:48.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.121+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f2b3011e860 con 0x7f2b380c3800 2023-10-30T12:52:48.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.121+0000 7f2b3db6f640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 0x7f2b1c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:48.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.121+0000 7f2b3db6f640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 0x7f2b1c0689e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b34001560 tx=0x7f2b34009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:48.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.125+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b380c2920 con 0x7f2b380c3800 2023-10-30T12:52:48.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b380c2920 con 0x7f2b380c3800 2023-10-30T12:52:48.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b2e7fc640 1 -- 172.21.15.8:0/3767793556 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f2b380c2920 con 0x7f2b380c3800 2023-10-30T12:52:48.318 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:48.319 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 msgr2=0x7f2b1c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 0x7f2b1c0689e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b34001560 tx=0x7f2b34009040 comp rx=0 tx=0).stop 2023-10-30T12:52:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 msgr2=0x7f2b3815fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:48.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.313+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b3001c760 tx=0x7f2b24005800 comp rx=0 tx=0).stop 2023-10-30T12:52:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 shutdown_connections 2023-10-30T12:52:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2b1c066520 0x7f2b1c0689e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b380c3800 0x7f2b3815fcb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2b38161dd0 0x7f2b380c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 --2- 172.21.15.8:0/3767793556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b24004f70 0x7f2b380c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 >> 172.21.15.8:0/3767793556 conn(0x7f2b380cac70 msgr2=0x7f2b381554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 shutdown_connections 2023-10-30T12:52:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:48.317+0000 7f2b3fdd3640 1 -- 172.21.15.8:0/3767793556 wait complete. 2023-10-30T12:52:48.339 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:52:50.340 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:50.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.485+0000 7f58204a9640 1 Processor -- start 2023-10-30T12:52:50.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.485+0000 7f58204a9640 1 -- start start 2023-10-30T12:52:50.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f58204a9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58180c5f80 0x7f58180c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f58204a9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f58180c9b40 0x7f58181597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f58204a9640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581815cda0 con 0x7f58180c5ba0 2023-10-30T12:52:50.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f58204a9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581815d990 con 0x7f58180c9b40 2023-10-30T12:52:50.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f58204a9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5818153f60 con 0x7f58180c5f80 2023-10-30T12:52:50.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581e245640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f58180c5ba0 0x7f5818069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43696/0 (socket says 172.21.15.8:43696) 2023-10-30T12:52:50.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581e245640 1 -- 172.21.15.8:0/4209022376 learned_addr learned my addr 172.21.15.8:0/4209022376 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:52:50.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581ea46640 1 --2- 172.21.15.8:0/4209022376 >> v2:172.21.15.8:3300/0 conn(0x7f58180c9b40 0x7f58181597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581da44640 1 --2- 172.21.15.8:0/4209022376 >> v2:172.21.15.46:3300/0 conn(0x7f58180c5f80 0x7f58180c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581ea46640 1 -- 172.21.15.8:0/4209022376 >> v2:172.21.15.46:3300/0 conn(0x7f58180c5f80 msgr2=0x7f58180c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:50.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581ea46640 1 --2- 172.21.15.8:0/4209022376 >> v2:172.21.15.46:3300/0 conn(0x7f58180c5f80 0x7f58180c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581ea46640 1 -- 172.21.15.8:0/4209022376 >> v1:172.21.15.8:6789/0 conn(0x7f58180c5ba0 legacy=0x7f5818069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.489+0000 7f581ea46640 1 -- 172.21.15.8:0/4209022376 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581806f490 con 0x7f58180c9b40 2023-10-30T12:52:50.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f581ea46640 1 --2- 172.21.15.8:0/4209022376 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c9b40 0x7f58181597b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f581806e290 tx=0x7f581001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6247a6a31335ed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:50.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f581d243640 1 -- 172.21.15.8:0/4209022376 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100a57d0 con 0x7f58180c9b40 2023-10-30T12:52:50.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f581d243640 1 -- 172.21.15.8:0/4209022376 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58100a5930 con 0x7f58180c9b40 2023-10-30T12:52:50.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f581d243640 1 -- 172.21.15.8:0/4209022376 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100a5cc0 con 0x7f58180c9b40 2023-10-30T12:52:50.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- 172.21.15.8:0/4209022376 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c9b40 msgr2=0x7f58181597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 --2- 172.21.15.8:0/4209022376 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c9b40 0x7f58181597b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f581806e290 tx=0x7f581001c910 comp rx=0 tx=0).stop 2023-10-30T12:52:50.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- 172.21.15.8:0/4209022376 shutdown_connections 2023-10-30T12:52:50.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 --2- 172.21.15.8:0/4209022376 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c9b40 0x7f58181597b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 --2- 172.21.15.8:0/4209022376 >> v2:172.21.15.46:3300/0 conn(0x7f58180c5f80 0x7f58180c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- 172.21.15.8:0/4209022376 >> 172.21.15.8:0/4209022376 conn(0x7f581806afa0 msgr2=0x7f581806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:50.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- 172.21.15.8:0/4209022376 shutdown_connections 2023-10-30T12:52:50.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- 172.21.15.8:0/4209022376 wait complete. 2023-10-30T12:52:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 Processor -- start 2023-10-30T12:52:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.493+0000 7f58204a9640 1 -- start start 2023-10-30T12:52:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58180c5ba0 0x7f58181f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 0x7f5818158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818154820 con 0x7f58180c5f80 2023-10-30T12:52:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58181ebc10 con 0x7f58180c9b40 2023-10-30T12:52:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58181ebef0 con 0x7f58180c5ba0 2023-10-30T12:52:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42462/0 (socket says 172.21.15.8:42462) 2023-10-30T12:52:50.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 -- 172.21.15.8:0/1119189035 learned_addr learned my addr 172.21.15.8:0/1119189035 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:50.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581e245640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58180c5ba0 0x7f58181f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581ea46640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 0x7f5818158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 -- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58180c5ba0 msgr2=0x7f58181f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58180c5ba0 0x7f58181f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 -- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 msgr2=0x7f5818158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 0x7f5818158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 -- 172.21.15.8:0/1119189035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810009a70 con 0x7f58180c5f80 2023-10-30T12:52:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581ea46640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 0x7f5818158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:50.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f581da44640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f580800d5d0 tx=0x7f580800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808004240 con 0x7f58180c5f80 2023-10-30T12:52:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080043a0 con 0x7f58180c5f80 2023-10-30T12:52:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808003840 con 0x7f58180c5f80 2023-10-30T12:52:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58181ec1d0 con 0x7f58180c5f80 2023-10-30T12:52:50.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58181ec790 con 0x7f58180c5f80 2023-10-30T12:52:50.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581804e170 con 0x7f58180c5f80 2023-10-30T12:52:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.497+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58080039a0 con 0x7f58180c5f80 2023-10-30T12:52:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.501+0000 7f580effd640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 0x7f57fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.501+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f58080842d0 con 0x7f58180c5f80 2023-10-30T12:52:50.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.501+0000 7f581e245640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 0x7f57fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:50.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.501+0000 7f581e245640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 0x7f57fc068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5814005890 tx=0x7f5814009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:50.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.505+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58080514f0 con 0x7f58180c5f80 2023-10-30T12:52:50.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58180c9650 con 0x7f58180c5f80 2023-10-30T12:52:50.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f580effd640 1 -- 172.21.15.8:0/1119189035 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f5808054b10 con 0x7f58180c5f80 2023-10-30T12:52:50.704 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:50.704 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:50.705 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:50.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 msgr2=0x7f57fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 0x7f57fc068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5814005890 tx=0x7f5814009040 comp rx=0 tx=0).stop 2023-10-30T12:52:50.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 msgr2=0x7f5818157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:50.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f580800d5d0 tx=0x7f580800da90 comp rx=0 tx=0).stop 2023-10-30T12:52:50.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 shutdown_connections 2023-10-30T12:52:50.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f57fc0664a0 0x7f57fc068960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58180c9b40 0x7f5818158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58180c5f80 0x7f5818157cb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 --2- 172.21.15.8:0/1119189035 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58180c5ba0 0x7f58181f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:50.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 >> 172.21.15.8:0/1119189035 conn(0x7f581806afa0 msgr2=0x7f58180733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:50.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 shutdown_connections 2023-10-30T12:52:50.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:50.701+0000 7f58204a9640 1 -- 172.21.15.8:0/1119189035 wait complete. 2023-10-30T12:52:50.725 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:52:52.726 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 Processor -- start 2023-10-30T12:52:52.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 -- start start 2023-10-30T12:52:52.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89d41516c0 0x7f89d40c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d40c82d0 0x7f89d40ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89d40cd4a0 0x7f89d40c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d40c74a0 con 0x7f89d41516c0 2023-10-30T12:52:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d4151aa0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89db405640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4158af0 con 0x7f89d40c82d0 2023-10-30T12:52:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89d91a1640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89d41516c0 0x7f89d40c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89d91a1640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89d41516c0 0x7f89d40c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42474/0 (socket says 172.21.15.8:42474) 2023-10-30T12:52:52.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89d91a1640 1 -- 172.21.15.8:0/1264862774 learned_addr learned my addr 172.21.15.8:0/1264862774 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89d89a0640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d40c82d0 0x7f89d40ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.877+0000 7f89d99a2640 1 --2- 172.21.15.8:0/1264862774 >> v2:172.21.15.46:3300/0 conn(0x7f89d40cd4a0 0x7f89d40c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 -- 172.21.15.8:0/1264862774 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d40c82d0 msgr2=0x7f89d40ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:52.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d40c82d0 0x7f89d40ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 -- 172.21.15.8:0/1264862774 >> v2:172.21.15.46:3300/0 conn(0x7f89d40cd4a0 msgr2=0x7f89d40c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1264862774 >> v2:172.21.15.46:3300/0 conn(0x7f89d40cd4a0 0x7f89d40c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 -- 172.21.15.8:0/1264862774 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4158510 con 0x7f89d41516c0 2023-10-30T12:52:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d41516c0 0x7f89d40c7d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89d4071f60 tx=0x7f89d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b8339c9daa265 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:52.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89cbfff640 1 -- 172.21.15.8:0/1264862774 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00ab600 con 0x7f89d41516c0 2023-10-30T12:52:52.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89cbfff640 1 -- 172.21.15.8:0/1264862774 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00a7050 con 0x7f89d41516c0 2023-10-30T12:52:52.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89cbfff640 1 -- 172.21.15.8:0/1264862774 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00ab980 con 0x7f89d41516c0 2023-10-30T12:52:52.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89db405640 1 -- 172.21.15.8:0/1264862774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d41516c0 msgr2=0x7f89d40c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:52.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.881+0000 7f89db405640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d41516c0 0x7f89d40c7d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89d4071f60 tx=0x7f89d001c500 comp rx=0 tx=0).stop 2023-10-30T12:52:52.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- 172.21.15.8:0/1264862774 shutdown_connections 2023-10-30T12:52:52.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- 172.21.15.8:0/1264862774 >> v2:172.21.15.46:3300/0 conn(0x7f89d40cd4a0 0x7f89d40c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d40c82d0 0x7f89d40ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- 172.21.15.8:0/1264862774 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d41516c0 0x7f89d40c7d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- 172.21.15.8:0/1264862774 >> 172.21.15.8:0/1264862774 conn(0x7f89d4071fb0 msgr2=0x7f89d40723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:52.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- 172.21.15.8:0/1264862774 shutdown_connections 2023-10-30T12:52:52.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- 172.21.15.8:0/1264862774 wait complete. 2023-10-30T12:52:52.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 Processor -- start 2023-10-30T12:52:52.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- start start 2023-10-30T12:52:52.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d40c82d0 0x7f89d4145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d41516c0 0x7f89d4148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41f0ba0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d414b750 con 0x7f89d40c82d0 2023-10-30T12:52:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89db405640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4146bf0 con 0x7f89d41516c0 2023-10-30T12:52:52.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42478/0 (socket says 172.21.15.8:42478) 2023-10-30T12:52:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 -- 172.21.15.8:0/1270314411 learned_addr learned my addr 172.21.15.8:0/1270314411 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d99a2640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d41516c0 0x7f89d4148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d40c82d0 0x7f89d4145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 -- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d41516c0 msgr2=0x7f89d4148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d41516c0 0x7f89d4148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 -- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d40c82d0 msgr2=0x7f89d4145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d40c82d0 0x7f89d4145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 -- 172.21.15.8:0/1270314411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00094d0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.885+0000 7f89d89a0640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e870 tx=0x7f89c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ee60 con 0x7f89d40cd4a0 2023-10-30T12:52:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400c940 con 0x7f89d40cd4a0 2023-10-30T12:52:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ccf0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d41eff20 con 0x7f89d40cd4a0 2023-10-30T12:52:52.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d41f04e0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d41f0ce0 con 0x7f89d40cd4a0 2023-10-30T12:52:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89c4013470 con 0x7f89d40cd4a0 2023-10-30T12:52:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 0x7f89b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f89c4084000 con 0x7f89d40cd4a0 2023-10-30T12:52:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.889+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 0x7f89b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:52.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.893+0000 7f89d91a1640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 0x7f89b8068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89d00ad000 tx=0x7f89d00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:52.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:52.893+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89c4051380 con 0x7f89d40cd4a0 2023-10-30T12:52:53.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.085+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89d404e170 con 0x7f89d40cd4a0 2023-10-30T12:52:53.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.085+0000 7f89c9ffb640 1 -- 172.21.15.8:0/1270314411 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f89c40549a0 con 0x7f89d40cd4a0 2023-10-30T12:52:53.090 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:53.090 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:53.090 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:53.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 msgr2=0x7f89b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 0x7f89b8068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89d00ad000 tx=0x7f89d00a6000 comp rx=0 tx=0).stop 2023-10-30T12:52:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 msgr2=0x7f89d4144070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e870 tx=0x7f89c400ed30 comp rx=0 tx=0).stop 2023-10-30T12:52:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 shutdown_connections 2023-10-30T12:52:53.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b80664a0 0x7f89b8068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89d41516c0 0x7f89d4148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89d40cd4a0 0x7f89d4144070 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 --2- 172.21.15.8:0/1270314411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d40c82d0 0x7f89d4145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 >> 172.21.15.8:0/1270314411 conn(0x7f89d4071fb0 msgr2=0x7f89d4156250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:53.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 shutdown_connections 2023-10-30T12:52:53.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:53.089+0000 7f89db405640 1 -- 172.21.15.8:0/1270314411 wait complete. 2023-10-30T12:52:53.111 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:52:55.112 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:55.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 Processor -- start 2023-10-30T12:52:55.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 -- start start 2023-10-30T12:52:55.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 0x7f59381597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593815a770 con 0x7f59380cb180 2023-10-30T12:52:55.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5938159cf0 con 0x7f5938153b50 2023-10-30T12:52:55.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593c93d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593806aa30 con 0x7f59380c62b0 2023-10-30T12:52:55.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f5936d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 0x7f59381597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:47104/0 (socket says 172.21.15.8:47104) 2023-10-30T12:52:55.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 -- 172.21.15.8:0/1732731854 learned_addr learned my addr 172.21.15.8:0/1732731854 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:55.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 -- 172.21.15.8:0/1732731854 >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 msgr2=0x7f59381597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:55.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 --2- 172.21.15.8:0/1732731854 >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 0x7f59381597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 -- 172.21.15.8:0/1732731854 >> v1:172.21.15.46:6789/0 conn(0x7f59380cb180 legacy=0x7f593806ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f593659c640 1 -- 172.21.15.8:0/1732731854 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380c9d40 con 0x7f59380c62b0 2023-10-30T12:52:55.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.269+0000 7f5936d9d640 1 --2- 172.21.15.8:0/1732731854 >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 0x7f59381597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:52:55.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593659c640 1 --2- 172.21.15.8:0/1732731854 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f592c009870 tx=0x7f592c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0cbf4c9f94252c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:55.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f5935d9b640 1 -- 172.21.15.8:0/1732731854 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0ab600 con 0x7f59380c62b0 2023-10-30T12:52:55.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f5935d9b640 1 -- 172.21.15.8:0/1732731854 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c0a7050 con 0x7f59380c62b0 2023-10-30T12:52:55.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f5935d9b640 1 -- 172.21.15.8:0/1732731854 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0ab980 con 0x7f59380c62b0 2023-10-30T12:52:55.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 -- 172.21.15.8:0/1732731854 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 msgr2=0x7f59380ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 --2- 172.21.15.8:0/1732731854 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f592c009870 tx=0x7f592c01c540 comp rx=0 tx=0).stop 2023-10-30T12:52:55.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 -- 172.21.15.8:0/1732731854 shutdown_connections 2023-10-30T12:52:55.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 --2- 172.21.15.8:0/1732731854 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f59380ca1d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 --2- 172.21.15.8:0/1732731854 >> v2:172.21.15.46:3300/0 conn(0x7f5938153b50 0x7f59381597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 -- 172.21.15.8:0/1732731854 >> 172.21.15.8:0/1732731854 conn(0x7f5938065a80 msgr2=0x7f5938065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:55.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 -- 172.21.15.8:0/1732731854 shutdown_connections 2023-10-30T12:52:55.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.273+0000 7f593c93d640 1 -- 172.21.15.8:0/1732731854 wait complete. 2023-10-30T12:52:55.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 Processor -- start 2023-10-30T12:52:55.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- start start 2023-10-30T12:52:55.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f5938154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938153b50 0x7f593814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59381f2770 con 0x7f59380cb180 2023-10-30T12:52:55.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5938159140 con 0x7f5938153b50 2023-10-30T12:52:55.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59381f21d0 con 0x7f59380c62b0 2023-10-30T12:52:55.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42494/0 (socket says 172.21.15.8:42494) 2023-10-30T12:52:55.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 -- 172.21.15.8:0/3285714459 learned_addr learned my addr 172.21.15.8:0/3285714459 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:55.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593759e640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938153b50 0x7f593814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f5936d9d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f5938154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 -- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 msgr2=0x7f5938154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f5938154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 -- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938153b50 msgr2=0x7f593814c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938153b50 0x7f593814c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 -- 172.21.15.8:0/3285714459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009420 con 0x7f59380cb180 2023-10-30T12:52:55.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593659c640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f592c01abf0 tx=0x7f592c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:55.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0b4420 con 0x7f59380cb180 2023-10-30T12:52:55.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c0b4580 con 0x7f59380cb180 2023-10-30T12:52:55.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0b48f0 con 0x7f59380cb180 2023-10-30T12:52:55.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c0094d0 con 0x7f59380cb180 2023-10-30T12:52:55.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.277+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5938158810 con 0x7f59380cb180 2023-10-30T12:52:55.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5938067540 con 0x7f59380cb180 2023-10-30T12:52:55.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f592c0b2070 con 0x7f59380cb180 2023-10-30T12:52:55.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f591f7fe640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 0x7f5924068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:55.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f592c11d140 con 0x7f59380cb180 2023-10-30T12:52:55.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f5936d9d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 0x7f5924068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:55.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.281+0000 7f5936d9d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 0x7f5924068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5928009be0 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:55.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.285+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f592c0ea410 con 0x7f59380cb180 2023-10-30T12:52:55.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.473+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5938148a60 con 0x7f59380cb180 2023-10-30T12:52:55.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.473+0000 7f591f7fe640 1 -- 172.21.15.8:0/3285714459 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f592c0eda30 con 0x7f59380cb180 2023-10-30T12:52:55.478 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:55.479 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:55.479 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:55.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 msgr2=0x7f5924068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 0x7f5924068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5928009be0 tx=0x7f5928008040 comp rx=0 tx=0).stop 2023-10-30T12:52:55.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 msgr2=0x7f5938147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:55.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f592c01abf0 tx=0x7f592c0192e0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 shutdown_connections 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59240664a0 0x7f5924068960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938153b50 0x7f593814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59380cb180 0x7f5938147d40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 --2- 172.21.15.8:0/3285714459 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59380c62b0 0x7f5938154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:55.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 >> 172.21.15.8:0/3285714459 conn(0x7f5938065a80 msgr2=0x7f593814e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:55.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 shutdown_connections 2023-10-30T12:52:55.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:55.477+0000 7f593c93d640 1 -- 172.21.15.8:0/3285714459 wait complete. 2023-10-30T12:52:55.500 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:52:57.502 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:52:57.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.652+0000 7fdf18091640 1 Processor -- start 2023-10-30T12:52:57.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.652+0000 7fdf18091640 1 -- start start 2023-10-30T12:52:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf10150830 0x7fdf1006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf100cb7d0 0x7fdf100c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf100c3e40 0x7fdf10070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf100c2310 con 0x7fdf100c3e40 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf10156710 con 0x7fdf100cb7d0 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf18091640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100c4360 con 0x7fdf10150830 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf100c3e40 0x7fdf10070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdf100c3e40 0x7fdf10070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42504/0 (socket says 172.21.15.8:42504) 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 -- 172.21.15.8:0/2890993744 learned_addr learned my addr 172.21.15.8:0/2890993744 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1562c640 1 --2- 172.21.15.8:0/2890993744 >> v2:172.21.15.46:3300/0 conn(0x7fdf100cb7d0 0x7fdf100c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf15e2d640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf10150830 0x7fdf1006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 -- 172.21.15.8:0/2890993744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf10150830 msgr2=0x7fdf1006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf10150830 0x7fdf1006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 -- 172.21.15.8:0/2890993744 >> v2:172.21.15.46:3300/0 conn(0x7fdf100cb7d0 msgr2=0x7fdf100c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:52:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 --2- 172.21.15.8:0/2890993744 >> v2:172.21.15.46:3300/0 conn(0x7fdf100cb7d0 0x7fdf100c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.656+0000 7fdf1662e640 1 -- 172.21.15.8:0/2890993744 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf101569b0 con 0x7fdf100c3e40 2023-10-30T12:52:57.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf1662e640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf10070520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf100cbd70 tx=0x7fdf0801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311570a82914396 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:57.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf14e2b640 1 -- 172.21.15.8:0/2890993744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080af400 con 0x7fdf100c3e40 2023-10-30T12:52:57.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf14e2b640 1 -- 172.21.15.8:0/2890993744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf080ab050 con 0x7fdf100c3e40 2023-10-30T12:52:57.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf14e2b640 1 -- 172.21.15.8:0/2890993744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080af780 con 0x7fdf100c3e40 2023-10-30T12:52:57.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 -- 172.21.15.8:0/2890993744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 msgr2=0x7fdf10070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:57.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf10070520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf100cbd70 tx=0x7fdf0801f1c0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 -- 172.21.15.8:0/2890993744 shutdown_connections 2023-10-30T12:52:57.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf10070520 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 --2- 172.21.15.8:0/2890993744 >> v2:172.21.15.46:3300/0 conn(0x7fdf100cb7d0 0x7fdf100c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 --2- 172.21.15.8:0/2890993744 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf10150830 0x7fdf1006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 -- 172.21.15.8:0/2890993744 >> 172.21.15.8:0/2890993744 conn(0x7fdf100cac70 msgr2=0x7fdf100cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:57.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 -- 172.21.15.8:0/2890993744 shutdown_connections 2023-10-30T12:52:57.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.660+0000 7fdf18091640 1 -- 172.21.15.8:0/2890993744 wait complete. 2023-10-30T12:52:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 Processor -- start 2023-10-30T12:52:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- start start 2023-10-30T12:52:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf100cb7d0 0x7fdf10146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 0x7fdf10155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10156580 con 0x7fdf100c3e40 2023-10-30T12:52:57.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10147a90 con 0x7fdf10150830 2023-10-30T12:52:57.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101f2cf0 con 0x7fdf100cb7d0 2023-10-30T12:52:57.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42512/0 (socket says 172.21.15.8:42512) 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 -- 172.21.15.8:0/646842458 learned_addr learned my addr 172.21.15.8:0/646842458 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf1662e640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 0x7fdf10155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf1562c640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf100cb7d0 0x7fdf10146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 -- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf100cb7d0 msgr2=0x7fdf10146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:57.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf100cb7d0 0x7fdf10146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 -- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 msgr2=0x7fdf10155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 0x7fdf10155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 -- 172.21.15.8:0/646842458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08009b30 con 0x7fdf100c3e40 2023-10-30T12:52:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf1662e640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 0x7fdf10155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:52:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf15e2d640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d800 tx=0x7fdf0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00b6c0 con 0x7fdf100c3e40 2023-10-30T12:52:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c00b820 con 0x7fdf100c3e40 2023-10-30T12:52:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00bb90 con 0x7fdf100c3e40 2023-10-30T12:52:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf101f3180 con 0x7fdf100c3e40 2023-10-30T12:52:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101f23c0 con 0x7fdf100c3e40 2023-10-30T12:52:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.664+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1004e170 con 0x7fdf100c3e40 2023-10-30T12:52:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.668+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf0c00bcf0 con 0x7fdf100c3e40 2023-10-30T12:52:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.668+0000 7fdf067fc640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 0x7fdef4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:52:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.668+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7fdf0c084660 con 0x7fdf100c3e40 2023-10-30T12:52:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.668+0000 7fdf1562c640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 0x7fdef4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:52:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.668+0000 7fdf1562c640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 0x7fdef4068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf101e95d0 tx=0x7fdf00008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:52:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.672+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf0c051890 con 0x7fdf100c3e40 2023-10-30T12:52:57.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf10070ac0 con 0x7fdf100c3e40 2023-10-30T12:52:57.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf067fc640 1 -- 172.21.15.8:0/646842458 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7fdf0c011340 con 0x7fdf100c3e40 2023-10-30T12:52:57.865 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:52:57.865 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:52:57.865 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:52:57.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 msgr2=0x7fdef4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 0x7fdef4068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf101e95d0 tx=0x7fdf00008040 comp rx=0 tx=0).stop 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 msgr2=0x7fdf1015a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.860+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d800 tx=0x7fdf0c00dcc0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 shutdown_connections 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdef40664a0 0x7fdef4068960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf10150830 0x7fdf10155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdf100cb7d0 0x7fdf10146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 --2- 172.21.15.8:0/646842458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf100c3e40 0x7fdf1015a130 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:52:57.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 >> 172.21.15.8:0/646842458 conn(0x7fdf100cac70 msgr2=0x7fdf100c5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:52:57.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 shutdown_connections 2023-10-30T12:52:57.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:52:57.864+0000 7fdf18091640 1 -- 172.21.15.8:0/646842458 wait complete. 2023-10-30T12:52:57.886 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:52:59.887 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:00.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 Processor -- start 2023-10-30T12:53:00.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 -- start start 2023-10-30T12:53:00.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c0c2aa0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c0c3ab0 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a72e9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0706e0 con 0x7f8a6c1537f0 2023-10-30T12:53:00.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a6bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a6bfff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38272/0 (socket says 172.21.15.8:38272) 2023-10-30T12:53:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.040+0000 7f8a6bfff640 1 -- 172.21.15.8:0/1810257186 learned_addr learned my addr 172.21.15.8:0/1810257186 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a70c38640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6bfff640 1 -- 172.21.15.8:0/1810257186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 msgr2=0x7f8a6c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6bfff640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6bfff640 1 -- 172.21.15.8:0/1810257186 >> v1:172.21.15.8:6789/0 conn(0x7f8a6c0cd4a0 legacy=0x7f8a6c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6bfff640 1 -- 172.21.15.8:0/1810257186 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0708e0 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6bfff640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b031e1b675130ff server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:00.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6b7fe640 1 -- 172.21.15.8:0/1810257186 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580a8a30 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6b7fe640 1 -- 172.21.15.8:0/1810257186 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a580a8b90 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.044+0000 7f8a6b7fe640 1 -- 172.21.15.8:0/1810257186 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580b0440 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- 172.21.15.8:0/1810257186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c15a2e0 msgr2=0x7f8a6c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5801ed30 comp rx=0 tx=0).stop 2023-10-30T12:53:00.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- 172.21.15.8:0/1810257186 shutdown_connections 2023-10-30T12:53:00.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c0ccf60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/1810257186 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- 172.21.15.8:0/1810257186 >> 172.21.15.8:0/1810257186 conn(0x7f8a6c071fb0 msgr2=0x7f8a6c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:00.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- 172.21.15.8:0/1810257186 shutdown_connections 2023-10-30T12:53:00.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- 172.21.15.8:0/1810257186 wait complete. 2023-10-30T12:53:00.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 Processor -- start 2023-10-30T12:53:00.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- start start 2023-10-30T12:53:00.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c1f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c1f2f80 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c157590 con 0x7f8a6c15a2e0 2023-10-30T12:53:00.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a72e9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c157870 con 0x7f8a6c1537f0 2023-10-30T12:53:00.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38286/0 (socket says 172.21.15.8:38286) 2023-10-30T12:53:00.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 -- 172.21.15.8:0/4234291549 learned_addr learned my addr 172.21.15.8:0/4234291549 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:00.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a6bfff640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c1f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a71439640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 -- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 msgr2=0x7f8a6c1f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c1f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 -- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a6c15a2e0 msgr2=0x7f8a6c159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.048+0000 7f8a70c38640 1 -- 172.21.15.8:0/4234291549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580094d0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a70c38640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d800 tx=0x7f8a6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:00.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000b6c0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000b820 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000bb90 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c157b50 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c158890 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c04e170 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8a6000bcf0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 0x7f8a50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:00.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6140+0+0 (secure 0 0 0) 0x7f8a600845d0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.052+0000 7f8a6bfff640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 0x7f8a50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:00.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.056+0000 7f8a6bfff640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 0x7f8a50068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a580b2000 tx=0x7f8a580aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:00.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.056+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8a60051890 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.248+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a6c0b2ff0 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.248+0000 7f8a697fa640 1 -- 172.21.15.8:0/4234291549 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6522 (secure 0 0 0) 0x7f8a60011340 con 0x7f8a6c0cd4a0 2023-10-30T12:53:00.253 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:00.253 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5266,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:52:44.153611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4303,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.8:6837/2926864154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2926864154},{"type":"v1","addr":"172.21.15.8:6837","nonce":2926864154}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2023-10-30T12:53:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 msgr2=0x7f8a50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 0x7f8a50068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a580b2000 tx=0x7f8a580aa070 comp rx=0 tx=0).stop 2023-10-30T12:53:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 msgr2=0x7f8a6c158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d800 tx=0x7f8a6000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 shutdown_connections 2023-10-30T12:53:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f8a500664a0 0x7f8a50068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a6c15a2e0 0x7f8a6c159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8a6c1537f0 0x7f8a6c1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 --2- 172.21.15.8:0/4234291549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a6c0cd4a0 0x7f8a6c158350 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 >> 172.21.15.8:0/4234291549 conn(0x7f8a6c071fb0 msgr2=0x7f8a6c14f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 shutdown_connections 2023-10-30T12:53:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:00.252+0000 7f8a72e9c640 1 -- 172.21.15.8:0/4234291549 wait complete. 2023-10-30T12:53:00.274 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-30T12:53:02.275 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:02.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.424+0000 7f5931b3e640 1 Processor -- start 2023-10-30T12:53:02.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.424+0000 7f5931b3e640 1 -- start start 2023-10-30T12:53:02.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f5931b3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f592c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f5931b3e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0c2310 con 0x7f592c0cb7d0 2023-10-30T12:53:02.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f5931b3e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c156710 con 0x7f592c070520 2023-10-30T12:53:02.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f5931b3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0c4360 con 0x7f592c150830 2023-10-30T12:53:02.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592affd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f592c0cb7d0 0x7f592c0c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55678/0 (socket says 172.21.15.8:55678) 2023-10-30T12:53:02.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592affd640 1 -- 172.21.15.8:0/3332319560 learned_addr learned my addr 172.21.15.8:0/3332319560 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:53:02.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592b7fe640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f592c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653003256 0 0) 0x7f592c0c2310 con 0x7f592c0cb7d0 2023-10-30T12:53:02.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5914005740 con 0x7f592c0cb7d0 2023-10-30T12:53:02.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150308024 0 0) 0x7f5914005740 con 0x7f592c0cb7d0 2023-10-30T12:53:02.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 msgr2=0x7f592c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:02.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f592c06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 >> v1:172.21.15.46:6789/0 conn(0x7f592c070520 legacy=0x7f592c160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c1569b0 con 0x7f592c0cb7d0 2023-10-30T12:53:02.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f5920001680 con 0x7f592c0cb7d0 2023-10-30T12:53:02.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 >> v1:172.21.15.8:6789/0 conn(0x7f592c0cb7d0 legacy=0x7f592c0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914005bc0 0x7f5914005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f59140069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5914006f20 0x7f59140073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0c4360 con 0x7f5914006f20 2023-10-30T12:53:02.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591400dce0 con 0x7f5914005bc0 2023-10-30T12:53:02.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591400dea0 con 0x7f592c150830 2023-10-30T12:53:02.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5914006f20 0x7f59140073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592affd640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f59140069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592b7fe640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914005bc0 0x7f5914005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 -- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 msgr2=0x7f59140069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:02.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c150830 0x7f59140069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 -- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914005bc0 msgr2=0x7f5914005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:02.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914005bc0 0x7f5914005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.428+0000 7f592bfff640 1 -- 172.21.15.8:0/3332319560 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914005740 con 0x7f5914006f20 2023-10-30T12:53:02.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f592bfff640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5914006f20 0x7f59140073a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5924001e40 tx=0x7f592401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eacbb2141b09d4d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924003b80 con 0x7f5914006f20 2023-10-30T12:53:02.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924003ce0 con 0x7f5914006f20 2023-10-30T12:53:02.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f592a7fc640 1 -- 172.21.15.8:0/3332319560 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240052c0 con 0x7f5914006f20 2023-10-30T12:53:02.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- 172.21.15.8:0/3332319560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5914006f20 msgr2=0x7f59140073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 --2- 172.21.15.8:0/3332319560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5914006f20 0x7f59140073a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5924001e40 tx=0x7f592401c740 comp rx=0 tx=0).stop 2023-10-30T12:53:02.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f592bfff640 1 -- 172.21.15.8:0/3332319560 reap_dead start 2023-10-30T12:53:02.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- 172.21.15.8:0/3332319560 shutdown_connections 2023-10-30T12:53:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- 172.21.15.8:0/3332319560 >> 172.21.15.8:0/3332319560 conn(0x7f592c0cac70 msgr2=0x7f592c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- 172.21.15.8:0/3332319560 shutdown_connections 2023-10-30T12:53:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- 172.21.15.8:0/3332319560 wait complete. 2023-10-30T12:53:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 Processor -- start 2023-10-30T12:53:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.432+0000 7f5931b3e640 1 -- start start 2023-10-30T12:53:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914006f20 0x7f592c0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c1535e0 0x7f592c147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1531c0 con 0x7f592c153fd0 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1f1d00 con 0x7f5914006f20 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0c8f90 con 0x7f592c1535e0 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38294/0 (socket says 172.21.15.8:38294) 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 -- 172.21.15.8:0/1637517342 learned_addr learned my addr 172.21.15.8:0/1637517342 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:02.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592b7fe640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914006f20 0x7f592c0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592bfff640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c1535e0 0x7f592c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 -- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c1535e0 msgr2=0x7f592c147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c1535e0 0x7f592c147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 -- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914006f20 msgr2=0x7f592c0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914006f20 0x7f592c0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 -- 172.21.15.8:0/1637517342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c152540 con 0x7f592c153fd0 2023-10-30T12:53:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f592affd640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f592000e7e0 tx=0x7f592000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000c870 con 0x7f592c153fd0 2023-10-30T12:53:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000c9f0 con 0x7f592c153fd0 2023-10-30T12:53:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c152820 con 0x7f592c153fd0 2023-10-30T12:53:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000cda0 con 0x7f592c153fd0 2023-10-30T12:53:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c152de0 con 0x7f592c153fd0 2023-10-30T12:53:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.436+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c152fa0 con 0x7f592c153fd0 2023-10-30T12:53:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.440+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f592001d460 con 0x7f592c153fd0 2023-10-30T12:53:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.440+0000 7f5913fff640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 0x7f5918068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.440+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f5920084170 con 0x7f592c153fd0 2023-10-30T12:53:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.440+0000 7f592b7fe640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 0x7f5918068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.440+0000 7f592b7fe640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 0x7f5918068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f591c009a30 tx=0x7f591c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.444+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59200512d0 con 0x7f592c153fd0 2023-10-30T12:53:02.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f592c066510 con 0x7f592c153fd0 2023-10-30T12:53:02.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5913fff640 1 -- 172.21.15.8:0/1637517342 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5734 (secure 0 0 0) 0x7f59200548f0 con 0x7f592c153fd0 2023-10-30T12:53:02.639 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:02.640 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:02.155629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:02.640 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 41 2023-10-30T12:53:02.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 msgr2=0x7f5918068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 0x7f5918068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f591c009a30 tx=0x7f591c008040 comp rx=0 tx=0).stop 2023-10-30T12:53:02.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 msgr2=0x7f592c14be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f592000e7e0 tx=0x7f592000eca0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 shutdown_connections 2023-10-30T12:53:02.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59180664a0 0x7f5918068960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f592c1535e0 0x7f592c147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f592c153fd0 0x7f592c14be60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 --2- 172.21.15.8:0/1637517342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5914006f20 0x7f592c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 >> 172.21.15.8:0/1637517342 conn(0x7f592c0cac70 msgr2=0x7f592c14ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:02.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 shutdown_connections 2023-10-30T12:53:02.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.636+0000 7f5931b3e640 1 -- 172.21.15.8:0/1637517342 wait complete. 2023-10-30T12:53:02.660 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-10-30T12:53:02.661 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:53:02.661 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:02.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 Processor -- start 2023-10-30T12:53:02.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 -- start start 2023-10-30T12:53:02.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa8f8153c80 0x7fa8f815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8f80c42a0 0x7fa8f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f8159f20 con 0x7fa8f80c4a10 2023-10-30T12:53:02.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f815dec0 con 0x7fa8f8153c80 2023-10-30T12:53:02.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8fda17640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f81599a0 con 0x7fa8f80c42a0 2023-10-30T12:53:02.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa8f8153c80 0x7fa8f815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa8f8153c80 0x7fa8f815cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38298/0 (socket says 172.21.15.8:38298) 2023-10-30T12:53:02.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3797159572 learned_addr learned my addr 172.21.15.8:0/3797159572 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:02.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f77fe640 1 --2- 172.21.15.8:0/3797159572 >> v2:172.21.15.46:3300/0 conn(0x7fa8f80c42a0 0x7fa8f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3797159572 >> v2:172.21.15.46:3300/0 conn(0x7fa8f80c42a0 msgr2=0x7fa8f80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:02.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 --2- 172.21.15.8:0/3797159572 >> v2:172.21.15.46:3300/0 conn(0x7fa8f80c42a0 0x7fa8f80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3797159572 >> v1:172.21.15.8:6789/0 conn(0x7fa8f80c4a10 legacy=0x7fa8f815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.800+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3797159572 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f815d620 con 0x7fa8f8153c80 2023-10-30T12:53:02.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8f6ffd640 1 --2- 172.21.15.8:0/3797159572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f8153c80 0x7fa8f815cf90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b1d40 tx=0x7fa8e801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeb8b85b07dff21 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8f67fc640 1 -- 172.21.15.8:0/3797159572 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80a89f0 con 0x7fa8f8153c80 2023-10-30T12:53:02.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8f67fc640 1 -- 172.21.15.8:0/3797159572 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e80a8b50 con 0x7fa8f8153c80 2023-10-30T12:53:02.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8f67fc640 1 -- 172.21.15.8:0/3797159572 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b0440 con 0x7fa8f8153c80 2023-10-30T12:53:02.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 -- 172.21.15.8:0/3797159572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f8153c80 msgr2=0x7fa8f815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3797159572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f8153c80 0x7fa8f815cf90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b1d40 tx=0x7fa8e801ecf0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 -- 172.21.15.8:0/3797159572 shutdown_connections 2023-10-30T12:53:02.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3797159572 >> v2:172.21.15.46:3300/0 conn(0x7fa8f80c42a0 0x7fa8f80ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3797159572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f8153c80 0x7fa8f815cf90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 -- 172.21.15.8:0/3797159572 >> 172.21.15.8:0/3797159572 conn(0x7fa8f80715c0 msgr2=0x7fa8f80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:02.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 -- 172.21.15.8:0/3797159572 shutdown_connections 2023-10-30T12:53:02.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.804+0000 7fa8fda17640 1 -- 172.21.15.8:0/3797159572 wait complete. 2023-10-30T12:53:02.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 Processor -- start 2023-10-30T12:53:02.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- start start 2023-10-30T12:53:02.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f80c4a10 0x7fa8f81f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa8f8153c80 0x7fa8f81f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f814f330 con 0x7fa8f80c42a0 2023-10-30T12:53:02.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f814f090 con 0x7fa8f80c4a10 2023-10-30T12:53:02.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f814f840 con 0x7fa8f8153c80 2023-10-30T12:53:02.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38306/0 (socket says 172.21.15.8:38306) 2023-10-30T12:53:02.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3254202076 learned_addr learned my addr 172.21.15.8:0/3254202076 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f77fe640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa8f8153c80 0x7fa8f81f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8effff640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f80c4a10 0x7fa8f81f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa8f8153c80 msgr2=0x7fa8f81f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa8f8153c80 0x7fa8f81f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f80c4a10 msgr2=0x7fa8f81f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f80c4a10 0x7fa8f81f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 -- 172.21.15.8:0/3254202076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80094d0 con 0x7fa8f80c42a0 2023-10-30T12:53:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f6ffd640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8000c10 tx=0x7fa8e801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b0ea0 con 0x7fa8f80c42a0 2023-10-30T12:53:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e80b9430 con 0x7fa8f80c42a0 2023-10-30T12:53:02.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b97c0 con 0x7fa8f80c42a0 2023-10-30T12:53:02.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f814fb20 con 0x7fa8f80c42a0 2023-10-30T12:53:02.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.808+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f81500e0 con 0x7fa8f80c42a0 2023-10-30T12:53:02.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f80c4710 con 0x7fa8f80c42a0 2023-10-30T12:53:02.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa8e80b7040 con 0x7fa8f80c42a0 2023-10-30T12:53:02.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8f4ff9640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 0x7fa8d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fa8e8122190 con 0x7fa8f80c42a0 2023-10-30T12:53:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8effff640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 0x7fa8d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.812+0000 7fa8effff640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 0x7fa8d8068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8152af0 tx=0x7fa8e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:02.816+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa8e80ef4a0 con 0x7fa8f80c42a0 2023-10-30T12:53:03.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.004+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8f81502d0 con 0x7fa8f80c42a0 2023-10-30T12:53:03.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.004+0000 7fa8f4ff9640 1 -- 172.21.15.8:0/3254202076 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5734 (secure 0 0 0) 0x7fa8e80f2ac0 con 0x7fa8f80c42a0 2023-10-30T12:53:03.010 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:03.011 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:02.155629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:03.011 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 41 2023-10-30T12:53:03.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 msgr2=0x7fa8d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 0x7fa8d8068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8152af0 tx=0x7fa8e0009040 comp rx=0 tx=0).stop 2023-10-30T12:53:03.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 msgr2=0x7fa8f8152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:03.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8000c10 tx=0x7fa8e801ba90 comp rx=0 tx=0).stop 2023-10-30T12:53:03.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 shutdown_connections 2023-10-30T12:53:03.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa8d80664a0 0x7fa8d8068960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f80c4a10 0x7fa8f81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa8f8153c80 0x7fa8f81f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 --2- 172.21.15.8:0/3254202076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8f80c42a0 0x7fa8f8152130 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:03.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 >> 172.21.15.8:0/3254202076 conn(0x7fa8f80715c0 msgr2=0x7fa8f8157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:03.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 shutdown_connections 2023-10-30T12:53:03.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:03.008+0000 7fa8fda17640 1 -- 172.21.15.8:0/3254202076 wait complete. 2023-10-30T12:53:03.032 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 41, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 33 }, "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 26 }, "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:53:02.155629+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5079, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:53:05.033 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 Processor -- start 2023-10-30T12:53:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 -- start start 2023-10-30T12:53:05.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f499c153c80 0x7f499c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f499c0c4a10 0x7f499c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c159f20 con 0x7f499c15a3b0 2023-10-30T12:53:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c15dec0 con 0x7f499c0c4a10 2023-10-30T12:53:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49a1301640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c1599a0 con 0x7f499c153c80 2023-10-30T12:53:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f499c0c4a10 0x7f499c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f499c0c4a10 0x7f499c0c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38318/0 (socket says 172.21.15.8:38318) 2023-10-30T12:53:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 -- 172.21.15.8:0/3037351654 learned_addr learned my addr 172.21.15.8:0/3037351654 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f499affd640 1 --2- 172.21.15.8:0/3037351654 >> v2:172.21.15.46:3300/0 conn(0x7f499c153c80 0x7f499c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 -- 172.21.15.8:0/3037351654 >> v2:172.21.15.46:3300/0 conn(0x7f499c153c80 msgr2=0x7f499c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 --2- 172.21.15.8:0/3037351654 >> v2:172.21.15.46:3300/0 conn(0x7f499c153c80 0x7f499c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 -- 172.21.15.8:0/3037351654 >> v1:172.21.15.46:6789/0 conn(0x7f499c15a3b0 legacy=0x7f499c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.184+0000 7f49927fc640 1 -- 172.21.15.8:0/3037351654 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0c47e0 con 0x7f499c0c4a10 2023-10-30T12:53:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49927fc640 1 --2- 172.21.15.8:0/3037351654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c0c4a10 0x7f499c0c42a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f499c0d00f0 tx=0x7f498c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d6fdc120016f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:05.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f499a7fc640 1 -- 172.21.15.8:0/3037351654 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0a89b0 con 0x7f499c0c4a10 2023-10-30T12:53:05.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f499a7fc640 1 -- 172.21.15.8:0/3037351654 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c0a8b10 con 0x7f499c0c4a10 2023-10-30T12:53:05.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f499a7fc640 1 -- 172.21.15.8:0/3037351654 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0a8ea0 con 0x7f499c0c4a10 2023-10-30T12:53:05.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 -- 172.21.15.8:0/3037351654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c0c4a10 msgr2=0x7f499c0c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:05.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 --2- 172.21.15.8:0/3037351654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c0c4a10 0x7f499c0c42a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f499c0d00f0 tx=0x7f498c01ecb0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 -- 172.21.15.8:0/3037351654 shutdown_connections 2023-10-30T12:53:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 --2- 172.21.15.8:0/3037351654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c0c4a10 0x7f499c0c42a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 --2- 172.21.15.8:0/3037351654 >> v2:172.21.15.46:3300/0 conn(0x7f499c153c80 0x7f499c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 -- 172.21.15.8:0/3037351654 >> 172.21.15.8:0/3037351654 conn(0x7f499c0715c0 msgr2=0x7f499c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 -- 172.21.15.8:0/3037351654 shutdown_connections 2023-10-30T12:53:05.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.188+0000 7f49a1301640 1 -- 172.21.15.8:0/3037351654 wait complete. 2023-10-30T12:53:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 Processor -- start 2023-10-30T12:53:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 -- start start 2023-10-30T12:53:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f499c0c4a10 0x7f499c1ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f499c15a3b0 0x7f499c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1f2e20 con 0x7f499c153c80 2023-10-30T12:53:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1ef930 con 0x7f499c15a3b0 2023-10-30T12:53:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1f2990 con 0x7f499c0c4a10 2023-10-30T12:53:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38330/0 (socket says 172.21.15.8:38330) 2023-10-30T12:53:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 -- 172.21.15.8:0/3402116317 learned_addr learned my addr 172.21.15.8:0/3402116317 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f499affd640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f499c0c4a10 0x7f499c1ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f499b7fe640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f499c15a3b0 0x7f499c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 -- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f499c0c4a10 msgr2=0x7f499c1ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:05.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f499c0c4a10 0x7f499c1ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 -- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f499c15a3b0 msgr2=0x7f499c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:05.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f499c15a3b0 0x7f499c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 -- 172.21.15.8:0/3402116317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0094d0 con 0x7f499c153c80 2023-10-30T12:53:05.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49927fc640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f498c0b3040 tx=0x7f498c01ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:05.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0b0e00 con 0x7f499c153c80 2023-10-30T12:53:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c0bb470 con 0x7f499c153c80 2023-10-30T12:53:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0bb820 con 0x7f499c153c80 2023-10-30T12:53:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.192+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c150ae0 con 0x7f499c153c80 2023-10-30T12:53:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c1f2060 con 0x7f499c153c80 2023-10-30T12:53:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04e170 con 0x7f499c153c80 2023-10-30T12:53:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f498c0b7070 con 0x7f499c153c80 2023-10-30T12:53:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f4993fff640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 0x7f497c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f498c122390 con 0x7f499c153c80 2023-10-30T12:53:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.196+0000 7f499affd640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 0x7f497c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.200+0000 7f499affd640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 0x7f497c068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f498400b9d0 tx=0x7f4984008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.200+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f498c0ef620 con 0x7f499c153c80 2023-10-30T12:53:05.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.396+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f499c04e170 con 0x7f499c153c80 2023-10-30T12:53:05.402 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.396+0000 7f4993fff640 1 -- 172.21.15.8:0/3402116317 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5734 (secure 0 0 0) 0x7f498c0f9030 con 0x7f499c153c80 2023-10-30T12:53:05.402 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:05.402 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:02.155629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:05.403 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 41 2023-10-30T12:53:05.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 msgr2=0x7f497c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:05.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 0x7f497c068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f498400b9d0 tx=0x7f4984008040 comp rx=0 tx=0).stop 2023-10-30T12:53:05.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 msgr2=0x7f499c1505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:05.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f498c0b3040 tx=0x7f498c01ba50 comp rx=0 tx=0).stop 2023-10-30T12:53:05.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 shutdown_connections 2023-10-30T12:53:05.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f497c0664a0 0x7f497c068960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f499c15a3b0 0x7f499c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f499c153c80 0x7f499c1505a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 --2- 172.21.15.8:0/3402116317 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f499c0c4a10 0x7f499c1ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:05.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 >> 172.21.15.8:0/3402116317 conn(0x7f499c0715c0 msgr2=0x7f499c152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:05.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 shutdown_connections 2023-10-30T12:53:05.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:05.400+0000 7f49a1301640 1 -- 172.21.15.8:0/3402116317 wait complete. 2023-10-30T12:53:05.424 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:07.425 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:07.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 Processor -- start 2023-10-30T12:53:07.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 -- start start 2023-10-30T12:53:07.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f12800c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1280071cb0 0x7f128015a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128015a770 con 0x7f1280153300 2023-10-30T12:53:07.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12800cca10 con 0x7f1280071cb0 2023-10-30T12:53:07.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f12854c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128015d0a0 con 0x7f12800718d0 2023-10-30T12:53:07.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1280071cb0 0x7f128015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1280071cb0 0x7f128015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38332/0 (socket says 172.21.15.8:38332) 2023-10-30T12:53:07.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 -- 172.21.15.8:0/173720435 learned_addr learned my addr 172.21.15.8:0/173720435 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:07.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127effd640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f12800c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 -- 172.21.15.8:0/173720435 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 msgr2=0x7f12800c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:07.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f12800c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 -- 172.21.15.8:0/173720435 >> v1:172.21.15.46:6789/0 conn(0x7f1280153300 legacy=0x7f12800cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.572+0000 7f127e7fc640 1 -- 172.21.15.8:0/173720435 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128015d380 con 0x7f1280071cb0 2023-10-30T12:53:07.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f127e7fc640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f128015a230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1274009870 tx=0x7f127401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e1027ced5375dc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:07.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f127dffb640 1 -- 172.21.15.8:0/173720435 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740a8a30 con 0x7f1280071cb0 2023-10-30T12:53:07.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f127dffb640 1 -- 172.21.15.8:0/173720435 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12740a8b90 con 0x7f1280071cb0 2023-10-30T12:53:07.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f127dffb640 1 -- 172.21.15.8:0/173720435 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740b0440 con 0x7f1280071cb0 2023-10-30T12:53:07.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- 172.21.15.8:0/173720435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 msgr2=0x7f128015a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f128015a230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1274009870 tx=0x7f127401ed30 comp rx=0 tx=0).stop 2023-10-30T12:53:07.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- 172.21.15.8:0/173720435 shutdown_connections 2023-10-30T12:53:07.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f128015a230 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 --2- 172.21.15.8:0/173720435 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f12800c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- 172.21.15.8:0/173720435 >> 172.21.15.8:0/173720435 conn(0x7f128006ab00 msgr2=0x7f1280065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:07.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- 172.21.15.8:0/173720435 shutdown_connections 2023-10-30T12:53:07.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- 172.21.15.8:0/173720435 wait complete. 2023-10-30T12:53:07.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 Processor -- start 2023-10-30T12:53:07.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.576+0000 7f12854c9640 1 -- start start 2023-10-30T12:53:07.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f128015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280153300 0x7f12801f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128014c590 con 0x7f1280071cb0 2023-10-30T12:53:07.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128015b420 con 0x7f1280153300 2023-10-30T12:53:07.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128015ca10 con 0x7f12800718d0 2023-10-30T12:53:07.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38346/0 (socket says 172.21.15.8:38346) 2023-10-30T12:53:07.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 -- 172.21.15.8:0/2084775145 learned_addr learned my addr 172.21.15.8:0/2084775145 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:07.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127effd640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f128015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127f7fe640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280153300 0x7f12801f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 -- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 msgr2=0x7f128015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f128015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 -- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280153300 msgr2=0x7f12801f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280153300 0x7f12801f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 -- 172.21.15.8:0/2084775145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740094d0 con 0x7f1280071cb0 2023-10-30T12:53:07.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f127e7fc640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f12740b3040 tx=0x7f127401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:07.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740b0dd0 con 0x7f1280071cb0 2023-10-30T12:53:07.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12740bb460 con 0x7f1280071cb0 2023-10-30T12:53:07.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128015b690 con 0x7f1280071cb0 2023-10-30T12:53:07.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740bb810 con 0x7f1280071cb0 2023-10-30T12:53:07.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128015bc50 con 0x7f1280071cb0 2023-10-30T12:53:07.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.580+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128004e170 con 0x7f1280071cb0 2023-10-30T12:53:07.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.584+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12740b83f0 con 0x7f1280071cb0 2023-10-30T12:53:07.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.584+0000 7f12677fe640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 0x7f126c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:07.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.584+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f12741219c0 con 0x7f1280071cb0 2023-10-30T12:53:07.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.584+0000 7f127effd640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 0x7f126c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:07.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.584+0000 7f127effd640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 0x7f126c068990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1270009b60 tx=0x7f1270009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:07.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.588+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f12740eed00 con 0x7f1280071cb0 2023-10-30T12:53:07.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.776+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128014c870 con 0x7f1280071cb0 2023-10-30T12:53:07.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.776+0000 7f12677fe640 1 -- 172.21.15.8:0/2084775145 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5734 (secure 0 0 0) 0x7f12740b5090 con 0x7f1280071cb0 2023-10-30T12:53:07.781 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:07.782 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:02.155629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:07.782 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 41 2023-10-30T12:53:07.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 msgr2=0x7f126c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 0x7f126c068990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1270009b60 tx=0x7f1270009190 comp rx=0 tx=0).stop 2023-10-30T12:53:07.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 msgr2=0x7f12800b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f12740b3040 tx=0x7f127401bad0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 shutdown_connections 2023-10-30T12:53:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f126c0664d0 0x7f126c068990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280153300 0x7f12801f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1280071cb0 0x7f12800b37c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 --2- 172.21.15.8:0/2084775145 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f12800718d0 0x7f128015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 >> 172.21.15.8:0/2084775145 conn(0x7f128006ab00 msgr2=0x7f1280065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 shutdown_connections 2023-10-30T12:53:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:07.780+0000 7f12854c9640 1 -- 172.21.15.8:0/2084775145 wait complete. 2023-10-30T12:53:07.803 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:09.804 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:09.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 Processor -- start 2023-10-30T12:53:09.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 -- start start 2023-10-30T12:53:09.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7ce00c8eb0 0x7f7ce015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 0x7f7ce015a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce006aeb0 0x7f7ce0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce0159800 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce015dcc0 con 0x7f7ce015ac70 2023-10-30T12:53:09.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7ce49f4640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce0159d80 con 0x7f7ce006aeb0 2023-10-30T12:53:09.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7ce00c8eb0 0x7f7ce015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7ce00c8eb0 0x7f7ce015cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40702/0 (socket says 172.21.15.8:40702) 2023-10-30T12:53:09.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 -- 172.21.15.8:0/1296985392 learned_addr learned my addr 172.21.15.8:0/1296985392 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:09.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cde59c640 1 --2- 172.21.15.8:0/1296985392 >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 0x7f7ce015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cdf59e640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce006aeb0 0x7f7ce0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 -- 172.21.15.8:0/1296985392 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce006aeb0 msgr2=0x7f7ce0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:09.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce006aeb0 0x7f7ce0069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 -- 172.21.15.8:0/1296985392 >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 msgr2=0x7f7ce015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:09.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 --2- 172.21.15.8:0/1296985392 >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 0x7f7ce015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cded9d640 1 -- 172.21.15.8:0/1296985392 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce015a750 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.956+0000 7f7cde59c640 1 --2- 172.21.15.8:0/1296985392 >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 0x7f7ce015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:09.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7cded9d640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce00c8eb0 0x7f7ce015cd70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00b4130 tx=0x7f7cd001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becfd86383d289e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:09.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7cddd9b640 1 -- 172.21.15.8:0/1296985392 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00a89b0 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7cddd9b640 1 -- 172.21.15.8:0/1296985392 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd00a8b10 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7cddd9b640 1 -- 172.21.15.8:0/1296985392 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00a8ea0 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 -- 172.21.15.8:0/1296985392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce00c8eb0 msgr2=0x7f7ce015cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:09.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce00c8eb0 0x7f7ce015cd70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00b4130 tx=0x7f7cd001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 -- 172.21.15.8:0/1296985392 shutdown_connections 2023-10-30T12:53:09.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce006aeb0 0x7f7ce0069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/1296985392 >> v2:172.21.15.46:3300/0 conn(0x7f7ce015ac70 0x7f7ce015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/1296985392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce00c8eb0 0x7f7ce015cd70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 -- 172.21.15.8:0/1296985392 >> 172.21.15.8:0/1296985392 conn(0x7f7ce00cef00 msgr2=0x7f7ce00cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:09.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 -- 172.21.15.8:0/1296985392 shutdown_connections 2023-10-30T12:53:09.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.960+0000 7f7ce49f4640 1 -- 172.21.15.8:0/1296985392 wait complete. 2023-10-30T12:53:09.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 Processor -- start 2023-10-30T12:53:09.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- start start 2023-10-30T12:53:09.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 0x7f7ce01f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce015ac70 0x7f7ce014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01ef700 con 0x7f7ce006aeb0 2023-10-30T12:53:09.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01f2e10 con 0x7f7ce015ac70 2023-10-30T12:53:09.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01f3070 con 0x7f7ce00c8eb0 2023-10-30T12:53:09.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40718/0 (socket says 172.21.15.8:40718) 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 -- 172.21.15.8:0/3683308594 learned_addr learned my addr 172.21.15.8:0/3683308594 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cdf59e640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce015ac70 0x7f7ce014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cde59c640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 0x7f7ce01f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 -- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 msgr2=0x7f7ce01f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 0x7f7ce01f0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 -- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce015ac70 msgr2=0x7f7ce014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce015ac70 0x7f7ce014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 -- 172.21.15.8:0/3683308594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00094d0 con 0x7f7ce006aeb0 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cde59c640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 0x7f7ce01f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cded9d640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001d3a0 tx=0x7f7cd001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00ba600 con 0x7f7ce006aeb0 2023-10-30T12:53:09.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd00ba760 con 0x7f7ce006aeb0 2023-10-30T12:53:09.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce01ef040 con 0x7f7ce006aeb0 2023-10-30T12:53:09.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00bab10 con 0x7f7ce006aeb0 2023-10-30T12:53:09.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce01eed10 con 0x7f7ce006aeb0 2023-10-30T12:53:09.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.964+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce01ef840 con 0x7f7ce006aeb0 2023-10-30T12:53:09.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.968+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7cd00bac70 con 0x7f7ce006aeb0 2023-10-30T12:53:09.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.968+0000 7f7cc77fe640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 0x7f7ccc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:09.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.968+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f7cd0122770 con 0x7f7ce006aeb0 2023-10-30T12:53:09.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.968+0000 7f7cde59c640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 0x7f7ccc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:09.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.968+0000 7f7cde59c640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 0x7f7ccc068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0152a70 tx=0x7f7cd4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:09.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:09.972+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7cd00ef960 con 0x7f7ce006aeb0 2023-10-30T12:53:10.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.160+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ce004e170 con 0x7f7ce006aeb0 2023-10-30T12:53:10.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.160+0000 7f7cc77fe640 1 -- 172.21.15.8:0/3683308594 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5736 (secure 0 0 0) 0x7f7cd00f2f80 con 0x7f7ce006aeb0 2023-10-30T12:53:10.165 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:10.166 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:08.324549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:resolve","state_seq":25,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:10.166 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 42 2023-10-30T12:53:10.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 msgr2=0x7f7ccc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:10.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 0x7f7ccc068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0152a70 tx=0x7f7cd4008040 comp rx=0 tx=0).stop 2023-10-30T12:53:10.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 msgr2=0x7f7ce00c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:10.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001d3a0 tx=0x7f7cd001ba90 comp rx=0 tx=0).stop 2023-10-30T12:53:10.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 shutdown_connections 2023-10-30T12:53:10.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7ccc0664a0 0x7f7ccc068960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:10.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce015ac70 0x7f7ce014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:10.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7ce00c8eb0 0x7f7ce01f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:10.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 --2- 172.21.15.8:0/3683308594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce006aeb0 0x7f7ce00c6af0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:10.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 >> 172.21.15.8:0/3683308594 conn(0x7f7ce00cef00 msgr2=0x7f7ce00c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:10.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 shutdown_connections 2023-10-30T12:53:10.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:10.164+0000 7f7ce49f4640 1 -- 172.21.15.8:0/3683308594 wait complete. 2023-10-30T12:53:10.187 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:12.187 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:12.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 Processor -- start 2023-10-30T12:53:12.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 -- start start 2023-10-30T12:53:12.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda6815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda68159f20 con 0x7fda680c4a10 2023-10-30T12:53:12.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6815d620 con 0x7fda680c42a0 2023-10-30T12:53:12.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6eef1640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815dec0 con 0x7fda68153c80 2023-10-30T12:53:12.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda67fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fda680c4a10 0x7fda6815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:46490/0 (socket says 172.21.15.8:46490) 2023-10-30T12:53:12.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda67fff640 1 -- 172.21.15.8:0/3557633512 learned_addr learned my addr 172.21.15.8:0/3557633512 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:53:12.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6cc8d640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda6815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90671353 0 0) 0x7fda68159f20 con 0x7fda680c4a10 2023-10-30T12:53:12.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda58003530 con 0x7fda680c4a10 2023-10-30T12:53:12.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117046826 0 0) 0x7fda58003530 con 0x7fda680c4a10 2023-10-30T12:53:12.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 msgr2=0x7fda6815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:12.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda6815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 >> v1:172.21.15.46:6789/0 conn(0x7fda680c42a0 legacy=0x7fda680ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6815d900 con 0x7fda680c4a10 2023-10-30T12:53:12.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fda54001680 con 0x7fda680c4a10 2023-10-30T12:53:12.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 >> v1:172.21.15.8:6789/0 conn(0x7fda680c4a10 legacy=0x7fda6815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58003ef0 0x7fda580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda58004d10 0x7fda58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815dec0 con 0x7fda58004d10 2023-10-30T12:53:12.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5800b9f0 con 0x7fda58003ef0 2023-10-30T12:53:12.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5800bbb0 con 0x7fda68153c80 2023-10-30T12:53:12.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6d48e640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda58004d10 0x7fda58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda6cc8d640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.340+0000 7fda67fff640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58003ef0 0x7fda580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 -- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 msgr2=0x7fda580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:12.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda68153c80 0x7fda580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 -- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58003ef0 msgr2=0x7fda580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:12.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58003ef0 0x7fda580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 -- 172.21.15.8:0/3557633512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58003530 con 0x7fda58004d10 2023-10-30T12:53:12.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda58004d10 0x7fda58005190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fda60001f60 tx=0x7fda6001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41539139a579b0ba server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:12.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600a5750 con 0x7fda58004d10 2023-10-30T12:53:12.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600a58b0 con 0x7fda58004d10 2023-10-30T12:53:12.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda677fe640 1 -- 172.21.15.8:0/3557633512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600a5ca0 con 0x7fda58004d10 2023-10-30T12:53:12.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 -- 172.21.15.8:0/3557633512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda58004d10 msgr2=0x7fda58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 --2- 172.21.15.8:0/3557633512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda58004d10 0x7fda58005190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fda60001f60 tx=0x7fda6001c860 comp rx=0 tx=0).stop 2023-10-30T12:53:12.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6d48e640 1 -- 172.21.15.8:0/3557633512 reap_dead start 2023-10-30T12:53:12.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 -- 172.21.15.8:0/3557633512 shutdown_connections 2023-10-30T12:53:12.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 -- 172.21.15.8:0/3557633512 >> 172.21.15.8:0/3557633512 conn(0x7fda680715c0 msgr2=0x7fda680719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:12.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 -- 172.21.15.8:0/3557633512 shutdown_connections 2023-10-30T12:53:12.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.344+0000 7fda6eef1640 1 -- 172.21.15.8:0/3557633512 wait complete. 2023-10-30T12:53:12.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 Processor -- start 2023-10-30T12:53:12.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 -- start start 2023-10-30T12:53:12.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58004d10 0x7fda680ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda680ce5d0 0x7fda68158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda68159550 con 0x7fda681590d0 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815af30 con 0x7fda58004d10 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6eef1640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda68149280 con 0x7fda680ce5d0 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40744/0 (socket says 172.21.15.8:40744) 2023-10-30T12:53:12.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 -- 172.21.15.8:0/89689731 learned_addr learned my addr 172.21.15.8:0/89689731 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:12.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6cc8d640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58004d10 0x7fda680ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda67fff640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda680ce5d0 0x7fda68158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 -- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda680ce5d0 msgr2=0x7fda68158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda680ce5d0 0x7fda68158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 -- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58004d10 msgr2=0x7fda680ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58004d10 0x7fda680ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 -- 172.21.15.8:0/89689731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680c9500 con 0x7fda681590d0 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda6d48e640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fda600b0040 tx=0x7fda58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600a5e30 con 0x7fda681590d0 2023-10-30T12:53:12.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600011d0 con 0x7fda681590d0 2023-10-30T12:53:12.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.348+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60003b80 con 0x7fda681590d0 2023-10-30T12:53:12.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6815ba40 con 0x7fda681590d0 2023-10-30T12:53:12.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6815b480 con 0x7fda681590d0 2023-10-30T12:53:12.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda681588b0 con 0x7fda681590d0 2023-10-30T12:53:12.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda600b4040 con 0x7fda681590d0 2023-10-30T12:53:12.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda657fa640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 0x7fda4c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:12.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fda6011d670 con 0x7fda681590d0 2023-10-30T12:53:12.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda6cc8d640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 0x7fda4c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:12.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.352+0000 7fda6cc8d640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 0x7fda4c068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009be0 tx=0x7fda5c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:12.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.356+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda681588b0 con 0x7fda681590d0 2023-10-30T12:53:12.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda681588b0 con 0x7fda681590d0 2023-10-30T12:53:12.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda657fa640 1 -- 172.21.15.8:0/89689731 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5736 (secure 0 0 0) 0x7fda681588b0 con 0x7fda681590d0 2023-10-30T12:53:12.552 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:12.552 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:08.324549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:resolve","state_seq":25,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:12.553 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 42 2023-10-30T12:53:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 msgr2=0x7fda4c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 0x7fda4c068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009be0 tx=0x7fda5c009190 comp rx=0 tx=0).stop 2023-10-30T12:53:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 msgr2=0x7fda68157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:12.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fda600b0040 tx=0x7fda58005800 comp rx=0 tx=0).stop 2023-10-30T12:53:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 shutdown_connections 2023-10-30T12:53:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda4c0664a0 0x7fda4c068960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda681590d0 0x7fda68157780 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda680ce5d0 0x7fda68158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 --2- 172.21.15.8:0/89689731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda58004d10 0x7fda680ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:12.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 >> 172.21.15.8:0/89689731 conn(0x7fda680715c0 msgr2=0x7fda680c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:12.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 shutdown_connections 2023-10-30T12:53:12.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:12.548+0000 7fda6eef1640 1 -- 172.21.15.8:0/89689731 wait complete. 2023-10-30T12:53:12.573 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:14.574 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.724+0000 7fdd1dc09640 1 Processor -- start 2023-10-30T12:53:14.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.724+0000 7fdd1dc09640 1 -- start start 2023-10-30T12:53:14.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd1dc09640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdd180cb7d0 0x7fdd180c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:14.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd1dc09640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd180c2310 con 0x7fdd180c3e40 2023-10-30T12:53:14.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd1dc09640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd18156710 con 0x7fdd18150830 2023-10-30T12:53:14.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd1dc09640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1806bc10 con 0x7fdd180cb7d0 2023-10-30T12:53:14.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdd180cb7d0 0x7fdd180c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:14.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdd180cb7d0 0x7fdd180c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40746/0 (socket says 172.21.15.8:40746) 2023-10-30T12:53:14.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 -- 172.21.15.8:0/3239193808 learned_addr learned my addr 172.21.15.8:0/3239193808 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:14.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 -- 172.21.15.8:0/3239193808 >> v1:172.21.15.46:6789/0 conn(0x7fdd18150830 legacy=0x7fdd1806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 -- 172.21.15.8:0/3239193808 >> v1:172.21.15.8:6789/0 conn(0x7fdd180c3e40 legacy=0x7fdd18070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.728+0000 7fdd0ffff640 1 -- 172.21.15.8:0/3239193808 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180c4360 con 0x7fdd180cb7d0 2023-10-30T12:53:14.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd0ffff640 1 --2- 172.21.15.8:0/3239193808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd180cb7d0 0x7fdd180c3900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd0001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0d6dab5900d794 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:14.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd16ffd640 1 -- 172.21.15.8:0/3239193808 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd000a8a30 con 0x7fdd180cb7d0 2023-10-30T12:53:14.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd16ffd640 1 -- 172.21.15.8:0/3239193808 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd000a8b90 con 0x7fdd180cb7d0 2023-10-30T12:53:14.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd16ffd640 1 -- 172.21.15.8:0/3239193808 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd000b0440 con 0x7fdd180cb7d0 2023-10-30T12:53:14.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- 172.21.15.8:0/3239193808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd180cb7d0 msgr2=0x7fdd180c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/3239193808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd180cb7d0 0x7fdd180c3900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd0001ed30 comp rx=0 tx=0).stop 2023-10-30T12:53:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- 172.21.15.8:0/3239193808 shutdown_connections 2023-10-30T12:53:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/3239193808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd180cb7d0 0x7fdd180c3900 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- 172.21.15.8:0/3239193808 >> 172.21.15.8:0/3239193808 conn(0x7fdd180cac70 msgr2=0x7fdd180cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- 172.21.15.8:0/3239193808 shutdown_connections 2023-10-30T12:53:14.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- 172.21.15.8:0/3239193808 wait complete. 2023-10-30T12:53:14.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 Processor -- start 2023-10-30T12:53:14.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- start start 2023-10-30T12:53:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdd180c3e40 0x7fdd181493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 0x7fdd18153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1814c720 con 0x7fdd18150830 2023-10-30T12:53:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1814c1a0 con 0x7fdd180cb7d0 2023-10-30T12:53:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd1dc09640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180c6c50 con 0x7fdd180c3e40 2023-10-30T12:53:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40758/0 (socket says 172.21.15.8:40758) 2023-10-30T12:53:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.732+0000 7fdd17fff640 1 -- 172.21.15.8:0/885546383 learned_addr learned my addr 172.21.15.8:0/885546383 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd0ffff640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 0x7fdd18153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd177fe640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdd180c3e40 0x7fdd181493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 -- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdd180c3e40 msgr2=0x7fdd181493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdd180c3e40 0x7fdd181493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 -- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 msgr2=0x7fdd18153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 0x7fdd18153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 -- 172.21.15.8:0/885546383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000094d0 con 0x7fdd18150830 2023-10-30T12:53:14.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd0ffff640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 0x7fdd18153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:14.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd17fff640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c710 tx=0x7fdd1000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:14.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10003ba0 con 0x7fdd18150830 2023-10-30T12:53:14.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd10003d00 con 0x7fdd18150830 2023-10-30T12:53:14.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000aa90 con 0x7fdd18150830 2023-10-30T12:53:14.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1814c440 con 0x7fdd18150830 2023-10-30T12:53:14.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd18152d80 con 0x7fdd18150830 2023-10-30T12:53:14.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1804e170 con 0x7fdd18150830 2023-10-30T12:53:14.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd1000abf0 con 0x7fdd18150830 2023-10-30T12:53:14.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 0x7fdcf8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:14.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.736+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fdd10083e70 con 0x7fdd18150830 2023-10-30T12:53:14.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.740+0000 7fdd177fe640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 0x7fdcf8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:14.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.740+0000 7fdd177fe640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 0x7fdcf8068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000f70 tx=0x7fdd08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:14.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.740+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdd100510f0 con 0x7fdd18150830 2023-10-30T12:53:14.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.932+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd18156490 con 0x7fdd18150830 2023-10-30T12:53:14.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.932+0000 7fdd14ff9640 1 -- 172.21.15.8:0/885546383 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5736 (secure 0 0 0) 0x7fdd10054710 con 0x7fdd18150830 2023-10-30T12:53:14.937 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:14.937 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:08.324549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:resolve","state_seq":25,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:14.938 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 42 2023-10-30T12:53:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 msgr2=0x7fdcf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 0x7fdcf8068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000f70 tx=0x7fdd08009040 comp rx=0 tx=0).stop 2023-10-30T12:53:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 msgr2=0x7fdd180c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:14.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c710 tx=0x7fdd1000cbd0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 shutdown_connections 2023-10-30T12:53:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdcf80664a0 0x7fdcf8068960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd180cb7d0 0x7fdd18153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdd18150830 0x7fdd180c6710 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 --2- 172.21.15.8:0/885546383 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdd180c3e40 0x7fdd181493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 >> 172.21.15.8:0/885546383 conn(0x7fdd180cac70 msgr2=0x7fdd18070520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 shutdown_connections 2023-10-30T12:53:14.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:14.936+0000 7fdd1dc09640 1 -- 172.21.15.8:0/885546383 wait complete. 2023-10-30T12:53:14.959 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 42, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 42, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4929": { "addr": "172.21.15.8:6839/3916953811", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 3916953811, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 3916953811, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 4929, "incarnation": 21, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 33 }, "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 26 }, "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:resolve", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:53:08.324549+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5079, "mds_2": 4929 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:53:15.444 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:15.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:16.960 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:17.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 Processor -- start 2023-10-30T12:53:17.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 -- start start 2023-10-30T12:53:17.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f868006bb40 0x7f8680157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f868006c420 0x7f8680157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8680071e00 con 0x7f868006b760 2023-10-30T12:53:17.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8680070020 con 0x7f868006c420 2023-10-30T12:53:17.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8685868640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8680070f10 con 0x7f868006bb40 2023-10-30T12:53:17.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f868006c420 0x7f8680157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f868006c420 0x7f8680157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40770/0 (socket says 172.21.15.8:40770) 2023-10-30T12:53:17.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 -- 172.21.15.8:0/2811189760 learned_addr learned my addr 172.21.15.8:0/2811189760 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:17.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f8677fff640 1 --2- 172.21.15.8:0/2811189760 >> v2:172.21.15.46:3300/0 conn(0x7f868006bb40 0x7f8680157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 -- 172.21.15.8:0/2811189760 >> v2:172.21.15.46:3300/0 conn(0x7f868006bb40 msgr2=0x7f8680157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:17.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 --2- 172.21.15.8:0/2811189760 >> v2:172.21.15.46:3300/0 conn(0x7f868006bb40 0x7f8680157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 -- 172.21.15.8:0/2811189760 >> v1:172.21.15.46:6789/0 conn(0x7f868006b760 legacy=0x7f868015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:17.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.112+0000 7f867f7fe640 1 -- 172.21.15.8:0/2811189760 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800711f0 con 0x7f868006c420 2023-10-30T12:53:17.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f867f7fe640 1 --2- 172.21.15.8:0/2811189760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006c420 0x7f8680157f60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f868006fa40 tx=0x7f867801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3accfaa9e4219a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:17.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f867e7fc640 1 -- 172.21.15.8:0/2811189760 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5820 con 0x7f868006c420 2023-10-30T12:53:17.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f867e7fc640 1 -- 172.21.15.8:0/2811189760 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780a5980 con 0x7f868006c420 2023-10-30T12:53:17.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f867e7fc640 1 -- 172.21.15.8:0/2811189760 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5d10 con 0x7f868006c420 2023-10-30T12:53:17.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 -- 172.21.15.8:0/2811189760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006c420 msgr2=0x7f8680157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:17.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 --2- 172.21.15.8:0/2811189760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006c420 0x7f8680157f60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f868006fa40 tx=0x7f867801c960 comp rx=0 tx=0).stop 2023-10-30T12:53:17.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 -- 172.21.15.8:0/2811189760 shutdown_connections 2023-10-30T12:53:17.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 --2- 172.21.15.8:0/2811189760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006c420 0x7f8680157f60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 --2- 172.21.15.8:0/2811189760 >> v2:172.21.15.46:3300/0 conn(0x7f868006bb40 0x7f8680157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 -- 172.21.15.8:0/2811189760 >> 172.21.15.8:0/2811189760 conn(0x7f86800cac70 msgr2=0x7f86800cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:17.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 -- 172.21.15.8:0/2811189760 shutdown_connections 2023-10-30T12:53:17.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.116+0000 7f8685868640 1 -- 172.21.15.8:0/2811189760 wait complete. 2023-10-30T12:53:17.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 Processor -- start 2023-10-30T12:53:17.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- start start 2023-10-30T12:53:17.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f868006bb40 0x7f86801591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f868006c420 0x7f8680159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86801616c0 con 0x7f868006b760 2023-10-30T12:53:17.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015b7b0 con 0x7f868006c420 2023-10-30T12:53:17.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8680160d60 con 0x7f868006bb40 2023-10-30T12:53:17.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40776/0 (socket says 172.21.15.8:40776) 2023-10-30T12:53:17.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 -- 172.21.15.8:0/2339387437 learned_addr learned my addr 172.21.15.8:0/2339387437 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:17.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867f7fe640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f868006c420 0x7f8680159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8677fff640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f868006bb40 0x7f86801591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 -- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f868006bb40 msgr2=0x7f86801591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:17.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f868006bb40 0x7f86801591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 -- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f868006c420 msgr2=0x7f8680159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:17.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f868006c420 0x7f8680159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 -- 172.21.15.8:0/2339387437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8678009b30 con 0x7f868006b760 2023-10-30T12:53:17.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867effd640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f867000c450 tx=0x7f867000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:17.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670012830 con 0x7f868006b760 2023-10-30T12:53:17.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700129b0 con 0x7f868006b760 2023-10-30T12:53:17.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670012d60 con 0x7f868006b760 2023-10-30T12:53:17.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868015b4d0 con 0x7f868006b760 2023-10-30T12:53:17.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.120+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801612e0 con 0x7f868006b760 2023-10-30T12:53:17.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680161800 con 0x7f868006b760 2023-10-30T12:53:17.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f867001c460 con 0x7f868006b760 2023-10-30T12:53:17.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f867cff9640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:17.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f8670083fe0 con 0x7f868006b760 2023-10-30T12:53:17.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f8677fff640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:17.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.124+0000 7f8677fff640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 0x7f8660068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86801534d0 tx=0x7f8668008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:17.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.128+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8680161800 con 0x7f868006b760 2023-10-30T12:53:17.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8680161800 con 0x7f868006b760 2023-10-30T12:53:17.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f867cff9640 1 -- 172.21.15.8:0/2339387437 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f8680161800 con 0x7f868006b760 2023-10-30T12:53:17.323 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:17.323 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:14.521001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:17.324 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 43 2023-10-30T12:53:17.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 msgr2=0x7f8660068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:17.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 0x7f8660068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86801534d0 tx=0x7f8668008040 comp rx=0 tx=0).stop 2023-10-30T12:53:17.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 msgr2=0x7f868015f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:17.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f867000c450 tx=0x7f867000c910 comp rx=0 tx=0).stop 2023-10-30T12:53:17.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 shutdown_connections 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86600664a0 0x7f8660068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f868006c420 0x7f8680159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f868006bb40 0x7f86801591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 --2- 172.21.15.8:0/2339387437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f868006b760 0x7f868015f9f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 >> 172.21.15.8:0/2339387437 conn(0x7f86800cac70 msgr2=0x7f86801577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:17.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 shutdown_connections 2023-10-30T12:53:17.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:17.320+0000 7f8685868640 1 -- 172.21.15.8:0/2339387437 wait complete. 2023-10-30T12:53:17.344 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:19.346 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:19.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 Processor -- start 2023-10-30T12:53:19.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 -- start start 2023-10-30T12:53:19.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8130151840 0x7f81300c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8130072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81300c5030 con 0x7f81300cb8b0 2023-10-30T12:53:19.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8130157830 con 0x7f8130151840 2023-10-30T12:53:19.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8136d65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8130157b10 con 0x7f8130151c20 2023-10-30T12:53:19.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8135302640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f81300cb8b0 0x7f813006b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:34926/0 (socket says 172.21.15.8:34926) 2023-10-30T12:53:19.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8135302640 1 -- 172.21.15.8:0/3230579736 learned_addr learned my addr 172.21.15.8:0/3230579736 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:53:19.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f812ffff640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8130072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f8134b01640 1 --2- 172.21.15.8:0/3230579736 >> v2:172.21.15.46:3300/0 conn(0x7f8130151840 0x7f81300c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609880547 0 0) 0x7f81300c5030 con 0x7f81300cb8b0 2023-10-30T12:53:19.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.496+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8120003530 con 0x7f81300cb8b0 2023-10-30T12:53:19.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779413150 0 0) 0x7f8120003530 con 0x7f81300cb8b0 2023-10-30T12:53:19.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 msgr2=0x7f8130072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:19.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8130072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 >> v2:172.21.15.46:3300/0 conn(0x7f8130151840 msgr2=0x7f81300c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:19.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 --2- 172.21.15.8:0/3230579736 >> v2:172.21.15.46:3300/0 conn(0x7f8130151840 0x7f81300c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130157df0 con 0x7f81300cb8b0 2023-10-30T12:53:19.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f8128001a00 con 0x7f81300cb8b0 2023-10-30T12:53:19.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 >> v1:172.21.15.8:6789/0 conn(0x7f81300cb8b0 legacy=0x7f813006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8120003980 0x7f8120003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200047d0 0x7f8120004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8120005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130157830 con 0x7f8120003980 2023-10-30T12:53:19.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130157b10 con 0x7f81200047d0 2023-10-30T12:53:19.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812000b9c0 con 0x7f8130151c20 2023-10-30T12:53:19.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8120003980 0x7f8120003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f812ffff640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200047d0 0x7f8120004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8135302640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8120005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 -- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 msgr2=0x7f8120005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:19.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8130151c20 0x7f8120005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 -- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200047d0 msgr2=0x7f8120004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:19.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200047d0 0x7f8120004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 -- 172.21.15.8:0/3230579736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120003530 con 0x7f8120003980 2023-10-30T12:53:19.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.500+0000 7f8134b01640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8120003980 0x7f8120003da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8124009ae0 tx=0x7f812401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f4edde76885957 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:19.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240a64a0 con 0x7f8120003980 2023-10-30T12:53:19.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81240a6600 con 0x7f8120003980 2023-10-30T12:53:19.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f812f7fe640 1 -- 172.21.15.8:0/3230579736 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240a69b0 con 0x7f8120003980 2023-10-30T12:53:19.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- 172.21.15.8:0/3230579736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8120003980 msgr2=0x7f8120003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 --2- 172.21.15.8:0/3230579736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8120003980 0x7f8120003da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8124009ae0 tx=0x7f812401ca90 comp rx=0 tx=0).stop 2023-10-30T12:53:19.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8135302640 1 -- 172.21.15.8:0/3230579736 reap_dead start 2023-10-30T12:53:19.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- 172.21.15.8:0/3230579736 shutdown_connections 2023-10-30T12:53:19.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- 172.21.15.8:0/3230579736 >> 172.21.15.8:0/3230579736 conn(0x7f81300cac70 msgr2=0x7f81300cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:19.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- 172.21.15.8:0/3230579736 shutdown_connections 2023-10-30T12:53:19.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- 172.21.15.8:0/3230579736 wait complete. 2023-10-30T12:53:19.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 Processor -- start 2023-10-30T12:53:19.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- start start 2023-10-30T12:53:19.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 0x7f81301dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f81301de810 0x7f81301dec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813014db00 con 0x7f813014baf0 2023-10-30T12:53:19.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301f2cb0 con 0x7f8120003980 2023-10-30T12:53:19.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.504+0000 7f8136d65640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813014d1e0 con 0x7f81301de810 2023-10-30T12:53:19.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58774/0 (socket says 172.21.15.8:58774) 2023-10-30T12:53:19.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 -- 172.21.15.8:0/2164208539 learned_addr learned my addr 172.21.15.8:0/2164208539 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:19.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8134b01640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 0x7f81301dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8135302640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f81301de810 0x7f81301dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 -- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f81301de810 msgr2=0x7f81301dec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f81301de810 0x7f81301dec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 -- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 msgr2=0x7f81301dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 0x7f81301dfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 -- 172.21.15.8:0/2164208539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81301f2820 con 0x7f813014baf0 2023-10-30T12:53:19.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8134b01640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 0x7f81301dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:19.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812ffff640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f811c0115c0 tx=0x7f811c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:19.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c004220 con 0x7f813014baf0 2023-10-30T12:53:19.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c004380 con 0x7f813014baf0 2023-10-30T12:53:19.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00f8f0 con 0x7f813014baf0 2023-10-30T12:53:19.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81301e03e0 con 0x7f813014baf0 2023-10-30T12:53:19.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130143f60 con 0x7f813014baf0 2023-10-30T12:53:19.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813004e170 con 0x7f813014baf0 2023-10-30T12:53:19.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.508+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f811c00fa50 con 0x7f813014baf0 2023-10-30T12:53:19.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.512+0000 7f812d7fa640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 0x7f8114068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:19.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.512+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f811c084380 con 0x7f813014baf0 2023-10-30T12:53:19.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.512+0000 7f8134b01640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 0x7f8114068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:19.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.512+0000 7f8134b01640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 0x7f8114068990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81240ad000 tx=0x7f81240b2040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:19.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.516+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f811c0514e0 con 0x7f813014baf0 2023-10-30T12:53:19.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.708+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8130155060 con 0x7f813014baf0 2023-10-30T12:53:19.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.708+0000 7f812d7fa640 1 -- 172.21.15.8:0/2164208539 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f811c054b00 con 0x7f813014baf0 2023-10-30T12:53:19.713 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:19.714 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:14.521001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:19.715 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 43 2023-10-30T12:53:19.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 msgr2=0x7f8114068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 0x7f8114068990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81240ad000 tx=0x7f81240b2040 comp rx=0 tx=0).stop 2023-10-30T12:53:19.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 msgr2=0x7f8130143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:19.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f811c0115c0 tx=0x7f811c011a80 comp rx=0 tx=0).stop 2023-10-30T12:53:19.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 shutdown_connections 2023-10-30T12:53:19.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f81140664d0 0x7f8114068990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f81301de810 0x7f81301dec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f813014baf0 0x7f8130143a20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 --2- 172.21.15.8:0/2164208539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8120003980 0x7f81301dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:19.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 >> 172.21.15.8:0/2164208539 conn(0x7f81300cac70 msgr2=0x7f81301458e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:19.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 shutdown_connections 2023-10-30T12:53:19.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:19.712+0000 7f8136d65640 1 -- 172.21.15.8:0/2164208539 wait complete. 2023-10-30T12:53:19.735 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:21.735 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:21.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 Processor -- start 2023-10-30T12:53:21.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 -- start start 2023-10-30T12:53:21.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f50a0153c80 0x7f50a015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a00c4a10 0x7f50a00c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 0x7f50a00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a015dec0 con 0x7f50a0153c80 2023-10-30T12:53:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a01599a0 con 0x7f50a015a3b0 2023-10-30T12:53:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a6846640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0159f20 con 0x7f50a00c4a10 2023-10-30T12:53:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f50a0153c80 0x7f50a015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f50a0153c80 0x7f50a015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58788/0 (socket says 172.21.15.8:58788) 2023-10-30T12:53:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 -- 172.21.15.8:0/2612577382 learned_addr learned my addr 172.21.15.8:0/2612577382 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:21.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509f7fe640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a00c4a10 0x7f50a00c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a4de3640 1 --2- 172.21.15.8:0/2612577382 >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 0x7f50a00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 -- 172.21.15.8:0/2612577382 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a00c4a10 msgr2=0x7f50a00c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:21.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a00c4a10 0x7f50a00c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 -- 172.21.15.8:0/2612577382 >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 msgr2=0x7f50a00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:21.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 --2- 172.21.15.8:0/2612577382 >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 0x7f50a00ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f509ffff640 1 -- 172.21.15.8:0/2612577382 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00c47e0 con 0x7f50a0153c80 2023-10-30T12:53:21.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.888+0000 7f50a4de3640 1 --2- 172.21.15.8:0/2612577382 >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 0x7f50a00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:53:21.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f509ffff640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a0153c80 0x7f50a015cf90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50a00cbbc0 tx=0x7f509001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca8d7e6cf4b557a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f509effd640 1 -- 172.21.15.8:0/2612577382 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900a89b0 con 0x7f50a0153c80 2023-10-30T12:53:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f509effd640 1 -- 172.21.15.8:0/2612577382 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50900a8b10 con 0x7f50a0153c80 2023-10-30T12:53:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f509effd640 1 -- 172.21.15.8:0/2612577382 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900a8ea0 con 0x7f50a0153c80 2023-10-30T12:53:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 -- 172.21.15.8:0/2612577382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a0153c80 msgr2=0x7f50a015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a0153c80 0x7f50a015cf90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50a00cbbc0 tx=0x7f509001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 -- 172.21.15.8:0/2612577382 shutdown_connections 2023-10-30T12:53:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 --2- 172.21.15.8:0/2612577382 >> v2:172.21.15.46:3300/0 conn(0x7f50a015a3b0 0x7f50a00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a00c4a10 0x7f50a00c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 --2- 172.21.15.8:0/2612577382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a0153c80 0x7f50a015cf90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 -- 172.21.15.8:0/2612577382 >> 172.21.15.8:0/2612577382 conn(0x7f50a00715c0 msgr2=0x7f50a00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 -- 172.21.15.8:0/2612577382 shutdown_connections 2023-10-30T12:53:21.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.892+0000 7f50a6846640 1 -- 172.21.15.8:0/2612577382 wait complete. 2023-10-30T12:53:21.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 Processor -- start 2023-10-30T12:53:21.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- start start 2023-10-30T12:53:21.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00c4a10 0x7f50a015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a0153c80 0x7f50a014df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015ab60 con 0x7f50a015a3b0 2023-10-30T12:53:21.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015b9c0 con 0x7f50a00c4a10 2023-10-30T12:53:21.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015adb0 con 0x7f50a0153c80 2023-10-30T12:53:21.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58790/0 (socket says 172.21.15.8:58790) 2023-10-30T12:53:21.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 -- 172.21.15.8:0/4239896714 learned_addr learned my addr 172.21.15.8:0/4239896714 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:21.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f509ffff640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00c4a10 0x7f50a015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f509f7fe640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a0153c80 0x7f50a014df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 -- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a0153c80 msgr2=0x7f50a014df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a0153c80 0x7f50a014df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 -- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00c4a10 msgr2=0x7f50a015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:21.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00c4a10 0x7f50a015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:21.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 -- 172.21.15.8:0/4239896714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900094d0 con 0x7f50a015a3b0 2023-10-30T12:53:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a4de3640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f509800cd80 tx=0x7f509800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800f680 con 0x7f50a015a3b0 2023-10-30T12:53:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800f800 con 0x7f50a015a3b0 2023-10-30T12:53:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800fbb0 con 0x7f50a015a3b0 2023-10-30T12:53:21.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a015b090 con 0x7f50a015a3b0 2023-10-30T12:53:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a015b650 con 0x7f50a015a3b0 2023-10-30T12:53:21.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.896+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a015bb50 con 0x7f50a015a3b0 2023-10-30T12:53:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.900+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f509800b040 con 0x7f50a015a3b0 2023-10-30T12:53:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.900+0000 7f509cff9640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 0x7f508c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:21.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.900+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f5098084a60 con 0x7f50a015a3b0 2023-10-30T12:53:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.900+0000 7f509ffff640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 0x7f508c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.900+0000 7f509ffff640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 0x7f508c068960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50900b2000 tx=0x7f5090009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:21.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:21.904+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5098051c70 con 0x7f50a015a3b0 2023-10-30T12:53:22.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.092+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50a004e170 con 0x7f50a015a3b0 2023-10-30T12:53:22.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.092+0000 7f509cff9640 1 -- 172.21.15.8:0/4239896714 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f5098055290 con 0x7f50a015a3b0 2023-10-30T12:53:22.097 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:22.097 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:20.330883+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:22.098 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 44 2023-10-30T12:53:22.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 msgr2=0x7f508c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:22.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 0x7f508c068960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50900b2000 tx=0x7f5090009040 comp rx=0 tx=0).stop 2023-10-30T12:53:22.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 msgr2=0x7f50a00c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:22.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f509800cd80 tx=0x7f509800a650 comp rx=0 tx=0).stop 2023-10-30T12:53:22.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 shutdown_connections 2023-10-30T12:53:22.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f508c0664a0 0x7f508c068960 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:22.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50a015a3b0 0x7f50a00c8e10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:22.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50a0153c80 0x7f50a014df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:22.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 --2- 172.21.15.8:0/4239896714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00c4a10 0x7f50a015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:22.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 >> 172.21.15.8:0/4239896714 conn(0x7f50a00715c0 msgr2=0x7f50a0158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:22.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 shutdown_connections 2023-10-30T12:53:22.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:22.096+0000 7f50a6846640 1 -- 172.21.15.8:0/4239896714 wait complete. 2023-10-30T12:53:22.119 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:24.120 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:24.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 Processor -- start 2023-10-30T12:53:24.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 -- start start 2023-10-30T12:53:24.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89c8151840 0x7f89c80c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89c8151c20 0x7f89c8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c80c5030 con 0x7f89c80cb8b0 2023-10-30T12:53:24.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c8157830 con 0x7f89c8151c20 2023-10-30T12:53:24.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89ce13e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c8157b10 con 0x7f89c8151840 2023-10-30T12:53:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89c8151c20 0x7f89c8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f89c8151c20 0x7f89c8072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58794/0 (socket says 172.21.15.8:58794) 2023-10-30T12:53:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 -- 172.21.15.8:0/22910323 learned_addr learned my addr 172.21.15.8:0/22910323 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c77fe640 1 --2- 172.21.15.8:0/22910323 >> v2:172.21.15.46:3300/0 conn(0x7f89c8151840 0x7f89c80c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 -- 172.21.15.8:0/22910323 >> v2:172.21.15.46:3300/0 conn(0x7f89c8151840 msgr2=0x7f89c80c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:24.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 --2- 172.21.15.8:0/22910323 >> v2:172.21.15.46:3300/0 conn(0x7f89c8151840 0x7f89c80c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 -- 172.21.15.8:0/22910323 >> v1:172.21.15.8:6789/0 conn(0x7f89c80cb8b0 legacy=0x7f89c806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.272+0000 7f89c6ffd640 1 -- 172.21.15.8:0/22910323 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c8157df0 con 0x7f89c8151c20 2023-10-30T12:53:24.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89c6ffd640 1 --2- 172.21.15.8:0/22910323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c8072b40 tx=0x7f89bc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c849a3e994438b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:24.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89c67fc640 1 -- 172.21.15.8:0/22910323 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0a89f0 con 0x7f89c8151c20 2023-10-30T12:53:24.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89c67fc640 1 -- 172.21.15.8:0/22910323 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc0a8b50 con 0x7f89c8151c20 2023-10-30T12:53:24.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89c67fc640 1 -- 172.21.15.8:0/22910323 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0b0440 con 0x7f89c8151c20 2023-10-30T12:53:24.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 -- 172.21.15.8:0/22910323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 msgr2=0x7f89c8072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 --2- 172.21.15.8:0/22910323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c8072b40 tx=0x7f89bc01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 -- 172.21.15.8:0/22910323 shutdown_connections 2023-10-30T12:53:24.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 --2- 172.21.15.8:0/22910323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072000 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 --2- 172.21.15.8:0/22910323 >> v2:172.21.15.46:3300/0 conn(0x7f89c8151840 0x7f89c80c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 -- 172.21.15.8:0/22910323 >> 172.21.15.8:0/22910323 conn(0x7f89c80cac70 msgr2=0x7f89c80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:24.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 -- 172.21.15.8:0/22910323 shutdown_connections 2023-10-30T12:53:24.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.276+0000 7f89ce13e640 1 -- 172.21.15.8:0/22910323 wait complete. 2023-10-30T12:53:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 Processor -- start 2023-10-30T12:53:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 -- start start 2023-10-30T12:53:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89c80cb8b0 0x7f89c814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89c8151840 0x7f89c814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8155b20 con 0x7f89c8151c20 2023-10-30T12:53:24.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c814dac0 con 0x7f89c80cb8b0 2023-10-30T12:53:24.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89ce13e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8157480 con 0x7f89c8151840 2023-10-30T12:53:24.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58796/0 (socket says 172.21.15.8:58796) 2023-10-30T12:53:24.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 -- 172.21.15.8:0/3035538518 learned_addr learned my addr 172.21.15.8:0/3035538518 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:24.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c77fe640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89c80cb8b0 0x7f89c814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c6ffd640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89c8151840 0x7f89c814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 -- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89c8151840 msgr2=0x7f89c814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89c8151840 0x7f89c814c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 -- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89c80cb8b0 msgr2=0x7f89c814cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89c80cb8b0 0x7f89c814cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 -- 172.21.15.8:0/3035538518 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0094d0 con 0x7f89c8151c20 2023-10-30T12:53:24.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89c7fff640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c7a0 tx=0x7f89c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:24.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0003b80 con 0x7f89c8151c20 2023-10-30T12:53:24.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.280+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c0003ce0 con 0x7f89c8151c20 2023-10-30T12:53:24.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c000aa30 con 0x7f89c8151c20 2023-10-30T12:53:24.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c814c910 con 0x7f89c8151c20 2023-10-30T12:53:24.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c8156b50 con 0x7f89c8151c20 2023-10-30T12:53:24.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c804e170 con 0x7f89c8151c20 2023-10-30T12:53:24.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89c000ab90 con 0x7f89c8151c20 2023-10-30T12:53:24.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89affff640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 0x7f89b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:24.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f89c0083ee0 con 0x7f89c8151c20 2023-10-30T12:53:24.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89c77fe640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 0x7f89b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:24.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.284+0000 7f89c77fe640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 0x7f89b4068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b9d0 tx=0x7f89b8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:24.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.288+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89c00510d0 con 0x7f89c8151c20 2023-10-30T12:53:24.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.480+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c81575c0 con 0x7f89c8151c20 2023-10-30T12:53:24.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89affff640 1 -- 172.21.15.8:0/3035538518 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f89c81575c0 con 0x7f89c8151c20 2023-10-30T12:53:24.487 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:24.487 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:20.330883+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:24.487 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 44 2023-10-30T12:53:24.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 msgr2=0x7f89b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 0x7f89b4068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b9d0 tx=0x7f89b8008040 comp rx=0 tx=0).stop 2023-10-30T12:53:24.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 msgr2=0x7f89c8072260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:24.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c7a0 tx=0x7f89c000cc60 comp rx=0 tx=0).stop 2023-10-30T12:53:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 shutdown_connections 2023-10-30T12:53:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89b40664a0 0x7f89b4068960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89c8151c20 0x7f89c8072260 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f89c8151840 0x7f89c814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 --2- 172.21.15.8:0/3035538518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89c80cb8b0 0x7f89c814cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 >> 172.21.15.8:0/3035538518 conn(0x7f89c80cac70 msgr2=0x7f89c80cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 shutdown_connections 2023-10-30T12:53:24.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:24.484+0000 7f89ce13e640 1 -- 172.21.15.8:0/3035538518 wait complete. 2023-10-30T12:53:24.508 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:53:26.509 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.660+0000 7f7dd2da7640 1 Processor -- start 2023-10-30T12:53:26.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.660+0000 7f7dd2da7640 1 -- start start 2023-10-30T12:53:26.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd2da7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dcc0c3100 0x7f7dcc0c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd2da7640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc0c39a0 con 0x7f7dcc159950 2023-10-30T12:53:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd2da7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc159f20 con 0x7f7dcc0c34e0 2023-10-30T12:53:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd2da7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc06d8d0 con 0x7f7dcc0c3100 2023-10-30T12:53:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd25a6640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7dcc159950 0x7f7dcc150240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:34946/0 (socket says 172.21.15.8:34946) 2023-10-30T12:53:26.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd25a6640 1 -- 172.21.15.8:0/2117133759 learned_addr learned my addr 172.21.15.8:0/2117133759 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:53:26.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd1da5640 1 --2- 172.21.15.8:0/2117133759 >> v2:172.21.15.46:3300/0 conn(0x7f7dcc0c3100 0x7f7dcc0c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214294050 0 0) 0x7f7dcc0c39a0 con 0x7f7dcc159950 2023-10-30T12:53:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db8003530 con 0x7f7dcc159950 2023-10-30T12:53:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866205418 0 0) 0x7f7db8003530 con 0x7f7dcc159950 2023-10-30T12:53:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 >> v2:172.21.15.46:3300/0 conn(0x7f7dcc0c3100 msgr2=0x7f7dcc0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:26.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 --2- 172.21.15.8:0/2117133759 >> v2:172.21.15.46:3300/0 conn(0x7f7dcc0c3100 0x7f7dcc0c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 >> v1:172.21.15.46:6789/0 conn(0x7f7dcc0c34e0 legacy=0x7f7dcc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc15a200 con 0x7f7dcc159950 2023-10-30T12:53:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f7dc4001a00 con 0x7f7dcc159950 2023-10-30T12:53:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 >> v1:172.21.15.8:6789/0 conn(0x7f7dcc159950 legacy=0x7f7dcc150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db8003980 0x7f7db8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 0x7f7db8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7db8005190 0x7f7db800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06d8d0 con 0x7f7db80047d0 2023-10-30T12:53:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db800c040 con 0x7f7db8003980 2023-10-30T12:53:26.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db800c200 con 0x7f7db8005190 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 0x7f7db8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd1da5640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db8003980 0x7f7db8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd25a6640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7db8005190 0x7f7db800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 -- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7db8005190 msgr2=0x7f7db800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7db8005190 0x7f7db800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 -- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db8003980 msgr2=0x7f7db8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db8003980 0x7f7db8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd15a4640 1 -- 172.21.15.8:0/2117133759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db8003530 con 0x7f7db80047d0 2023-10-30T12:53:26.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.664+0000 7f7dd25a6640 1 -- 172.21.15.8:0/2117133759 reap_dead start 2023-10-30T12:53:26.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 0x7f7db8004c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc001780 tx=0x7f7dbc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431fc6d6a9f088f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:26.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0052d0 con 0x7f7db80047d0 2023-10-30T12:53:26.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc0a7050 con 0x7f7db80047d0 2023-10-30T12:53:26.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd0da3640 1 -- 172.21.15.8:0/2117133759 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc003600 con 0x7f7db80047d0 2023-10-30T12:53:26.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 -- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 msgr2=0x7f7db8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 0x7f7db8004c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc001780 tx=0x7f7dbc01c3e0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 -- 172.21.15.8:0/2117133759 shutdown_connections 2023-10-30T12:53:26.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/2117133759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7db80047d0 0x7f7db8004c50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 -- 172.21.15.8:0/2117133759 >> 172.21.15.8:0/2117133759 conn(0x7f7dcc0caae0 msgr2=0x7f7dcc0caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:26.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 -- 172.21.15.8:0/2117133759 shutdown_connections 2023-10-30T12:53:26.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.668+0000 7f7dd2da7640 1 -- 172.21.15.8:0/2117133759 wait complete. 2023-10-30T12:53:26.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 Processor -- start 2023-10-30T12:53:26.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- start start 2023-10-30T12:53:26.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80047d0 0x7f7dcc1eb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 0x7f7dcc151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0ba360 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0b9810 con 0x7f7db80047d0 2023-10-30T12:53:26.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0b8690 con 0x7f7dcc1516b0 2023-10-30T12:53:26.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58806/0 (socket says 172.21.15.8:58806) 2023-10-30T12:53:26.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 -- 172.21.15.8:0/3934964996 learned_addr learned my addr 172.21.15.8:0/3934964996 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:26.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd1da5640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80047d0 0x7f7dcc1eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 0x7f7dcc151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 -- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 msgr2=0x7f7dcc151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 0x7f7dcc151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 -- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80047d0 msgr2=0x7f7dcc1eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80047d0 0x7f7dcc1eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 -- 172.21.15.8:0/3934964996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc152420 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd15a4640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 0x7f7dcc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd25a6640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159d30 tx=0x7f7dc4007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400aab0 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc400ac10 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc152010 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40133f0 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc152980 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.672+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc1eb5b0 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.676+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7dc4013550 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.676+0000 7f7dc27fc640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:26.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.676+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f7dc4084740 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.676+0000 7f7dd1da5640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:26.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.676+0000 7f7dd1da5640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 0x7f7db0068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc04f480 tx=0x7f7dc8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:26.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.680+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7dc40518a0 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dcc04e170 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dc27fc640 1 -- 172.21.15.8:0/3934964996 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f7dc4054ec0 con 0x7f7dcc0b8e20 2023-10-30T12:53:26.879 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:26.879 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:25.380761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:26.880 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 45 2023-10-30T12:53:26.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 msgr2=0x7f7db0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 0x7f7db0068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc04f480 tx=0x7f7dc8009190 comp rx=0 tx=0).stop 2023-10-30T12:53:26.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 msgr2=0x7f7dcc0b92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159d30 tx=0x7f7dc4007b10 comp rx=0 tx=0).stop 2023-10-30T12:53:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 shutdown_connections 2023-10-30T12:53:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dcc0b8e20 0x7f7dcc0b92d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7dcc1516b0 0x7f7dcc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 --2- 172.21.15.8:0/3934964996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80047d0 0x7f7dcc1eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:26.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 >> 172.21.15.8:0/3934964996 conn(0x7f7dcc0caae0 msgr2=0x7f7dcc0c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:26.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 shutdown_connections 2023-10-30T12:53:26.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:26.876+0000 7f7dd2da7640 1 -- 172.21.15.8:0/3934964996 wait complete. 2023-10-30T12:53:26.900 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=4303 2023-10-30T12:53:26.900 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2023-10-30T12:53:28.901 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-10-30T12:53:28.901 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-30T12:53:28.902 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-30T12:53:28.906 INFO:tasks.ceph.mds.a:Started 2023-10-30T12:53:28.906 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:29.004 INFO:tasks.ceph.mds.a.smithi008.stdout:starting mds.a at 2023-10-30T12:53:29.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 Processor -- start 2023-10-30T12:53:29.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 -- start start 2023-10-30T12:53:29.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9938153c80 0x7f993815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9938159f20 con 0x7f99380c42a0 2023-10-30T12:53:29.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993815d620 con 0x7f99380c4a10 2023-10-30T12:53:29.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f993db3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993815dec0 con 0x7f9938153c80 2023-10-30T12:53:29.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f9937fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f99380c42a0 0x7f99380ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:34964/0 (socket says 172.21.15.8:34964) 2023-10-30T12:53:29.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f9937fff640 1 -- 172.21.15.8:0/3213779143 learned_addr learned my addr 172.21.15.8:0/3213779143 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:53:29.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99377fe640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9938153c80 0x7f993815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477280076 0 0) 0x7f9938159f20 con 0x7f99380c42a0 2023-10-30T12:53:29.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9920003530 con 0x7f99380c42a0 2023-10-30T12:53:29.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219393060 0 0) 0x7f9920003530 con 0x7f99380c42a0 2023-10-30T12:53:29.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9938153c80 msgr2=0x7f993815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:29.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9938153c80 0x7f993815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 >> v1:172.21.15.46:6789/0 conn(0x7f99380c4a10 legacy=0x7f993815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993815d900 con 0x7f99380c42a0 2023-10-30T12:53:29.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f9930001a00 con 0x7f99380c42a0 2023-10-30T12:53:29.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 >> v1:172.21.15.8:6789/0 conn(0x7f99380c42a0 legacy=0x7f99380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99200039b0 0x7f9920003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.060+0000 7f99367fc640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f9920004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9920005030 0x7f992000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993815dec0 con 0x7f9920004360 2023-10-30T12:53:29.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992000bed0 con 0x7f99200039b0 2023-10-30T12:53:29.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992000c010 con 0x7f9920005030 2023-10-30T12:53:29.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99377fe640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9938153c80 0x7f993815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:53:29.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f9920004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9937fff640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9920005030 0x7f992000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99377fe640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99200039b0 0x7f9920003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 -- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9920005030 msgr2=0x7f992000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:29.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9920005030 0x7f992000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 -- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99200039b0 msgr2=0x7f9920003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:29.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99200039b0 0x7f9920003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 -- 172.21.15.8:0/3213779143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920003530 con 0x7f9920004360 2023-10-30T12:53:29.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f9936ffd640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f9920004830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f99380c4e30 tx=0x7f992c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d8f597ce642a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0ab600 con 0x7f9920004360 2023-10-30T12:53:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0a7050 con 0x7f9920004360 2023-10-30T12:53:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.064+0000 7f99367fc640 1 -- 172.21.15.8:0/3213779143 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0ab9a0 con 0x7f9920004360 2023-10-30T12:53:29.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- 172.21.15.8:0/3213779143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 msgr2=0x7f9920004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 --2- 172.21.15.8:0/3213779143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f9920004830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f99380c4e30 tx=0x7f992c01c520 comp rx=0 tx=0).stop 2023-10-30T12:53:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f9937fff640 1 -- 172.21.15.8:0/3213779143 reap_dead start 2023-10-30T12:53:29.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- 172.21.15.8:0/3213779143 shutdown_connections 2023-10-30T12:53:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- 172.21.15.8:0/3213779143 >> 172.21.15.8:0/3213779143 conn(0x7f99380715c0 msgr2=0x7f99380719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- 172.21.15.8:0/3213779143 shutdown_connections 2023-10-30T12:53:29.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- 172.21.15.8:0/3213779143 wait complete. 2023-10-30T12:53:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 Processor -- start 2023-10-30T12:53:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- start start 2023-10-30T12:53:29.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f993815ab60 0x7f9938147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99381f0d30 0x7f99381f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381f0930 con 0x7f9920004360 2023-10-30T12:53:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381f1720 con 0x7f99381f0d30 2023-10-30T12:53:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f993db3e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380cef00 con 0x7f993815ab60 2023-10-30T12:53:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58830/0 (socket says 172.21.15.8:58830) 2023-10-30T12:53:29.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 -- 172.21.15.8:0/642059213 learned_addr learned my addr 172.21.15.8:0/642059213 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:29.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f9936ffd640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f993815ab60 0x7f9938147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f9937fff640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99381f0d30 0x7f99381f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 -- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f993815ab60 msgr2=0x7f9938147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f993815ab60 0x7f9938147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 -- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99381f0d30 msgr2=0x7f99381f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99381f0d30 0x7f99381f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.068+0000 7f99377fe640 1 -- 172.21.15.8:0/642059213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993815c4c0 con 0x7f9920004360 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f99377fe640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9928011800 tx=0x7f9928011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800f9c0 con 0x7f9920004360 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800fb20 con 0x7f9920004360 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800fed0 con 0x7f9920004360 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993815c7a0 con 0x7f9920004360 2023-10-30T12:53:29.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99381f0550 con 0x7f9920004360 2023-10-30T12:53:29.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993804e170 con 0x7f9920004360 2023-10-30T12:53:29.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9928015460 con 0x7f9920004360 2023-10-30T12:53:29.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 0x7f9924068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:29.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f9928084710 con 0x7f9920004360 2023-10-30T12:53:29.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.072+0000 7f9936ffd640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 0x7f9924068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:29.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.076+0000 7f9936ffd640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 0x7f9924068990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f992c0ad000 tx=0x7f992c001330 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:29.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.076+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9928051900 con 0x7f9920004360 2023-10-30T12:53:29.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.268+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99380cad40 con 0x7f9920004360 2023-10-30T12:53:29.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.268+0000 7f991ffff640 1 -- 172.21.15.8:0/642059213 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f9928054f20 con 0x7f9920004360 2023-10-30T12:53:29.273 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:29.273 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:25.380761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:29.274 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 45 2023-10-30T12:53:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 msgr2=0x7f9924068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 0x7f9924068990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f992c0ad000 tx=0x7f992c001330 comp rx=0 tx=0).stop 2023-10-30T12:53:29.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 msgr2=0x7f99380ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9928011800 tx=0x7f9928011cc0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 shutdown_connections 2023-10-30T12:53:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f99240664d0 0x7f9924068990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99381f0d30 0x7f99381f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f993815ab60 0x7f9938147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 --2- 172.21.15.8:0/642059213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9920004360 0x7f99380ce9c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 >> 172.21.15.8:0/642059213 conn(0x7f99380715c0 msgr2=0x7f99380cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 shutdown_connections 2023-10-30T12:53:29.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:29.272+0000 7f993db3e640 1 -- 172.21.15.8:0/642059213 wait complete. 2023-10-30T12:53:29.295 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-10-30T12:53:31.296 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:53:31.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 Processor -- start 2023-10-30T12:53:31.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 -- start start 2023-10-30T12:53:31.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda4406c420 0x7fda440c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda440c7170 0x7fda44159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440c5f80 con 0x7fda44153bb0 2023-10-30T12:53:31.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440c7580 con 0x7fda440c7170 2023-10-30T12:53:31.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4be9c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440c5690 con 0x7fda4406c420 2023-10-30T12:53:31.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4a439640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda440c7170 0x7fda44159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda49437640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda4406c420 0x7fda440c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4a439640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda440c7170 0x7fda44159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41512/0 (socket says 172.21.15.8:41512) 2023-10-30T12:53:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.456+0000 7fda4a439640 1 -- 172.21.15.8:0/781203017 learned_addr learned my addr 172.21.15.8:0/781203017 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4a439640 1 -- 172.21.15.8:0/781203017 >> v2:172.21.15.46:3300/0 conn(0x7fda4406c420 msgr2=0x7fda440c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:53:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4a439640 1 --2- 172.21.15.8:0/781203017 >> v2:172.21.15.46:3300/0 conn(0x7fda4406c420 0x7fda440c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4a439640 1 -- 172.21.15.8:0/781203017 >> v1:172.21.15.46:6789/0 conn(0x7fda44153bb0 legacy=0x7fda440c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4a439640 1 -- 172.21.15.8:0/781203017 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4415a580 con 0x7fda440c7170 2023-10-30T12:53:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4a439640 1 --2- 172.21.15.8:0/781203017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda44159980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda4406f800 tx=0x7fda3c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16243462ed79b16 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda48c36640 1 -- 172.21.15.8:0/781203017 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0af400 con 0x7fda440c7170 2023-10-30T12:53:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda48c36640 1 -- 172.21.15.8:0/781203017 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c0ab050 con 0x7fda440c7170 2023-10-30T12:53:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda48c36640 1 -- 172.21.15.8:0/781203017 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0af780 con 0x7fda440c7170 2023-10-30T12:53:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 -- 172.21.15.8:0/781203017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 msgr2=0x7fda44159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 --2- 172.21.15.8:0/781203017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda44159980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda4406f800 tx=0x7fda3c01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 -- 172.21.15.8:0/781203017 shutdown_connections 2023-10-30T12:53:31.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 --2- 172.21.15.8:0/781203017 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda44159980 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 --2- 172.21.15.8:0/781203017 >> v2:172.21.15.46:3300/0 conn(0x7fda4406c420 0x7fda440c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.460+0000 7fda4be9c640 1 -- 172.21.15.8:0/781203017 >> 172.21.15.8:0/781203017 conn(0x7fda440cac70 msgr2=0x7fda440cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- 172.21.15.8:0/781203017 shutdown_connections 2023-10-30T12:53:31.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- 172.21.15.8:0/781203017 wait complete. 2023-10-30T12:53:31.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 Processor -- start 2023-10-30T12:53:31.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- start start 2023-10-30T12:53:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda4406c420 0x7fda44070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 0x7fda44154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c4e20 con 0x7fda440c7170 2023-10-30T12:53:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c3f70 con 0x7fda44153bb0 2023-10-30T12:53:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4be9c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c4210 con 0x7fda4406c420 2023-10-30T12:53:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41524/0 (socket says 172.21.15.8:41524) 2023-10-30T12:53:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 -- 172.21.15.8:0/1153516197 learned_addr learned my addr 172.21.15.8:0/1153516197 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:53:31.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49c38640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda4406c420 0x7fda44070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4a439640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 0x7fda44154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 -- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda4406c420 msgr2=0x7fda44070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda4406c420 0x7fda44070e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 -- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 msgr2=0x7fda44154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 0x7fda44154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda49437640 1 -- 172.21.15.8:0/1153516197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009b30 con 0x7fda440c7170 2023-10-30T12:53:31.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.464+0000 7fda4a439640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 0x7fda44154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:53:31.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda49437640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e870 tx=0x7fda3400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:31.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34009510 con 0x7fda440c7170 2023-10-30T12:53:31.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3400eec0 con 0x7fda440c7170 2023-10-30T12:53:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440c44f0 con 0x7fda440c7170 2023-10-30T12:53:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400cb40 con 0x7fda440c7170 2023-10-30T12:53:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440c4ab0 con 0x7fda440c7170 2023-10-30T12:53:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44154b50 con 0x7fda440c7170 2023-10-30T12:53:31.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda3400cca0 con 0x7fda440c7170 2023-10-30T12:53:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:53:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fda340840a0 con 0x7fda440c7170 2023-10-30T12:53:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.468+0000 7fda49c38640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:53:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.472+0000 7fda49c38640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 0x7fda28068990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda40001810 tx=0x7fda40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:53:31.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.472+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda340513d0 con 0x7fda440c7170 2023-10-30T12:53:31.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.664+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda4404e170 con 0x7fda440c7170 2023-10-30T12:53:31.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.664+0000 7fda3a7fc640 1 -- 172.21.15.8:0/1153516197 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+6523 (secure 0 0 0) 0x7fda340549f0 con 0x7fda440c7170 2023-10-30T12:53:31.669 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:53:31.669 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:53:25.380761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:53:31.670 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 46 2023-10-30T12:53:31.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 msgr2=0x7fda28068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 0x7fda28068990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda40001810 tx=0x7fda40009040 comp rx=0 tx=0).stop 2023-10-30T12:53:31.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 msgr2=0x7fda441565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:53:31.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e870 tx=0x7fda3400ec70 comp rx=0 tx=0).stop 2023-10-30T12:53:31.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 shutdown_connections 2023-10-30T12:53:31.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda44153bb0 0x7fda44154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda440c7170 0x7fda441565b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 --2- 172.21.15.8:0/1153516197 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda4406c420 0x7fda44070e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:53:31.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 >> 172.21.15.8:0/1153516197 conn(0x7fda440cac70 msgr2=0x7fda44070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:53:31.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 shutdown_connections 2023-10-30T12:53:31.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:53:31.668+0000 7fda4be9c640 1 -- 172.21.15.8:0/1153516197 wait complete. 2023-10-30T12:53:31.690 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-10-30T12:53:31.690 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 85 secs before thrashing 2023-10-30T12:53:32.074 INFO:tasks.workunit.client.0.smithi008.stderr:error: RPC failed; curl 56 GnuTLS recv error (-110): The TLS connection was non-properly terminated. 2023-10-30T12:53:32.074 INFO:tasks.workunit.client.0.smithi008.stderr:fetch-pack: unexpected disconnect while reading sideband packet 2023-10-30T12:53:32.076 INFO:tasks.workunit.client.0.smithi008.stderr:fatal: early EOF 2023-10-30T12:53:32.076 INFO:tasks.workunit.client.0.smithi008.stderr:fatal: fetch-pack: invalid index-pack output 2023-10-30T12:53:32.317 INFO:tasks.workunit.client.0.smithi008.stderr:+ retry 2023-10-30T12:53:32.317 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm -rf ceph 2023-10-30T12:53:32.318 INFO:tasks.workunit.client.0.smithi008.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2023-10-30T12:53:32.324 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into 'ceph'... 2023-10-30T12:53:34.403 INFO:tasks.ceph.mds.f.smithi046.stderr:2023-10-30T12:53:34.399+0000 7f93c5536640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-30T12:53:34.403 INFO:tasks.ceph.mds.f.smithi046.stderr:2023-10-30T12:53:34.399+0000 7f93c5536640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-30T12:53:34.403 INFO:tasks.ceph.mds.f.smithi046.stderr:2023-10-30T12:53:34.399+0000 7f93c5536640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-30T12:53:45.493 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:53:45.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:15.551 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:15.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:45.616 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:45.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:54:56.691 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:56.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 Processor -- start 2023-10-30T12:54:56.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 -- start start 2023-10-30T12:54:56.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f116c1516c0 0x7f116c0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0c82d0 0x7f116c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c151aa0 con 0x7f116c0cd4a0 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c0c74a0 con 0x7f116c1516c0 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f1171e54640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c158510 con 0x7f116c0c82d0 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f116c1516c0 0x7f116c0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f116c1516c0 0x7f116c0c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43874/0 (socket says 172.21.15.8:43874) 2023-10-30T12:54:56.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 -- 172.21.15.8:0/2509100071 learned_addr learned my addr 172.21.15.8:0/2509100071 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:54:56.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116affd640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0c82d0 0x7f116c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 -- 172.21.15.8:0/2509100071 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0c82d0 msgr2=0x7f116c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:56.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0c82d0 0x7f116c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:56.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 -- 172.21.15.8:0/2509100071 >> v1:172.21.15.46:6789/0 conn(0x7f116c0cd4a0 legacy=0x7f116c0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:56.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.843+0000 7f116b7fe640 1 -- 172.21.15.8:0/2509100071 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c158af0 con 0x7f116c1516c0 2023-10-30T12:54:56.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f116b7fe640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c1516c0 0x7f116c0c7d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f115c009870 tx=0x7f115c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6ac473376652f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:56.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f116a7fc640 1 -- 172.21.15.8:0/2509100071 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0ac410 con 0x7f116c1516c0 2023-10-30T12:54:56.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f116a7fc640 1 -- 172.21.15.8:0/2509100071 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c0ac570 con 0x7f116c1516c0 2023-10-30T12:54:56.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f116a7fc640 1 -- 172.21.15.8:0/2509100071 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0ac900 con 0x7f116c1516c0 2023-10-30T12:54:56.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 -- 172.21.15.8:0/2509100071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c1516c0 msgr2=0x7f116c0c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:56.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c1516c0 0x7f116c0c7d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f115c009870 tx=0x7f115c01c500 comp rx=0 tx=0).stop 2023-10-30T12:54:56.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 -- 172.21.15.8:0/2509100071 shutdown_connections 2023-10-30T12:54:56.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0c82d0 0x7f116c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:56.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 --2- 172.21.15.8:0/2509100071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c1516c0 0x7f116c0c7d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:56.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 -- 172.21.15.8:0/2509100071 >> 172.21.15.8:0/2509100071 conn(0x7f116c071fb0 msgr2=0x7f116c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:56.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 -- 172.21.15.8:0/2509100071 shutdown_connections 2023-10-30T12:54:56.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.847+0000 7f1171e54640 1 -- 172.21.15.8:0/2509100071 wait complete. 2023-10-30T12:54:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 Processor -- start 2023-10-30T12:54:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- start start 2023-10-30T12:54:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0cd4a0 0x7f116c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c1516c0 0x7f116c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0c9650 con 0x7f116c0c82d0 2023-10-30T12:54:56.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0ca4f0 con 0x7f116c1516c0 2023-10-30T12:54:56.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1f5470 con 0x7f116c0cd4a0 2023-10-30T12:54:56.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43888/0 (socket says 172.21.15.8:43888) 2023-10-30T12:54:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 -- 172.21.15.8:0/3318108042 learned_addr learned my addr 172.21.15.8:0/3318108042 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:54:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116affd640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0cd4a0 0x7f116c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116bfff640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c1516c0 0x7f116c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 -- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0cd4a0 msgr2=0x7f116c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0cd4a0 0x7f116c0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 -- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c1516c0 msgr2=0x7f116c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c1516c0 0x7f116c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 -- 172.21.15.8:0/3318108042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0094d0 con 0x7f116c0c82d0 2023-10-30T12:54:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f116b7fe640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f115c0af040 tx=0x7f115c0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:56.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0b66b0 con 0x7f116c0c82d0 2023-10-30T12:54:56.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c0b6810 con 0x7f116c0c82d0 2023-10-30T12:54:56.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0b6bc0 con 0x7f116c0c82d0 2023-10-30T12:54:56.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1f5750 con 0x7f116c0c82d0 2023-10-30T12:54:56.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.851+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c1f5d10 con 0x7f116c0c82d0 2023-10-30T12:54:56.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c04e170 con 0x7f116c0c82d0 2023-10-30T12:54:56.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f115c0b6d20 con 0x7f116c0c82d0 2023-10-30T12:54:56.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f1153fff640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 0x7f1158068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:56.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f115c11ebe0 con 0x7f116c0c82d0 2023-10-30T12:54:56.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f116affd640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 0x7f1158068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:56.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.855+0000 7f116affd640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 0x7f1158068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f116c1575d0 tx=0x7f1160008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:56.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:56.859+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f115c0ebe60 con 0x7f116c0c82d0 2023-10-30T12:54:57.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.051+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f116c160870 con 0x7f116c0c82d0 2023-10-30T12:54:57.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.051+0000 7f1153fff640 1 -- 172.21.15.8:0/3318108042 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f115c0b1090 con 0x7f116c0c82d0 2023-10-30T12:54:57.056 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:54:57.057 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:57.057 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:54:57.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.051+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 msgr2=0x7f1158068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 0x7f1158068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f116c1575d0 tx=0x7f1160008040 comp rx=0 tx=0).stop 2023-10-30T12:54:57.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 msgr2=0x7f116c148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f115c0af040 tx=0x7f115c0192a0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 shutdown_connections 2023-10-30T12:54:57.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1158066470 0x7f1158068930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c1516c0 0x7f116c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f116c0cd4a0 0x7f116c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 --2- 172.21.15.8:0/3318108042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f116c0c82d0 0x7f116c148e20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 >> 172.21.15.8:0/3318108042 conn(0x7f116c071fb0 msgr2=0x7f116c159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:57.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 shutdown_connections 2023-10-30T12:54:57.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.055+0000 7f1171e54640 1 -- 172.21.15.8:0/3318108042 wait complete. 2023-10-30T12:54:57.078 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2023-10-30T12:54:57.078 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:54:57.079 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:54:57.095 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:54:57.095 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:57.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 Processor -- start 2023-10-30T12:54:57.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 -- start start 2023-10-30T12:54:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0e4151840 0x7ff0e40c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151c20 0x7ff0e4072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff0e40cb8b0 0x7ff0e4072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40c5030 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e4157830 con 0x7ff0e4151840 2023-10-30T12:54:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.275+0000 7ff0eb49c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4157b10 con 0x7ff0e4151c20 2023-10-30T12:54:57.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff0e40cb8b0 0x7ff0e4072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff0e40cb8b0 0x7ff0e4072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43894/0 (socket says 172.21.15.8:43894) 2023-10-30T12:54:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 -- 172.21.15.8:0/276945652 learned_addr learned my addr 172.21.15.8:0/276945652 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:54:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9238640 1 --2- 172.21.15.8:0/276945652 >> v2:172.21.15.46:3300/0 conn(0x7ff0e4151840 0x7ff0e40c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e8a37640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151c20 0x7ff0e4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 -- 172.21.15.8:0/276945652 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151c20 msgr2=0x7ff0e4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151c20 0x7ff0e4072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 -- 172.21.15.8:0/276945652 >> v2:172.21.15.46:3300/0 conn(0x7ff0e4151840 msgr2=0x7ff0e40c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 --2- 172.21.15.8:0/276945652 >> v2:172.21.15.46:3300/0 conn(0x7ff0e4151840 0x7ff0e40c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 -- 172.21.15.8:0/276945652 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4157df0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0e9a39640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e4072540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4072b40 tx=0x7ff0dc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87850b9e4194e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0dbfff640 1 -- 172.21.15.8:0/276945652 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0af400 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0dbfff640 1 -- 172.21.15.8:0/276945652 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0ab050 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.279+0000 7ff0dbfff640 1 -- 172.21.15.8:0/276945652 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0af780 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- 172.21.15.8:0/276945652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 msgr2=0x7ff0e4072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e4072540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4072b40 tx=0x7ff0dc01f1c0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- 172.21.15.8:0/276945652 shutdown_connections 2023-10-30T12:54:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e4072540 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/276945652 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151c20 0x7ff0e4072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/276945652 >> v2:172.21.15.46:3300/0 conn(0x7ff0e4151840 0x7ff0e40c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- 172.21.15.8:0/276945652 >> 172.21.15.8:0/276945652 conn(0x7ff0e40cac70 msgr2=0x7ff0e40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- 172.21.15.8:0/276945652 shutdown_connections 2023-10-30T12:54:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- 172.21.15.8:0/276945652 wait complete. 2023-10-30T12:54:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 Processor -- start 2023-10-30T12:54:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- start start 2023-10-30T12:54:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151840 0x7ff0e414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e4151c20 0x7ff0e4142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e41efa30 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e41eeeb0 con 0x7ff0e4151c20 2023-10-30T12:54:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0eb49c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e41ef190 con 0x7ff0e4151840 2023-10-30T12:54:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43906/0 (socket says 172.21.15.8:43906) 2023-10-30T12:54:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 -- 172.21.15.8:0/2739960905 learned_addr learned my addr 172.21.15.8:0/2739960905 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:54:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e8a37640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151840 0x7ff0e414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9a39640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e4151c20 0x7ff0e4142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 -- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151840 msgr2=0x7ff0e414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151840 0x7ff0e414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 -- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e4151c20 msgr2=0x7ff0e4142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e4151c20 0x7ff0e4142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.283+0000 7ff0e9238640 1 -- 172.21.15.8:0/2739960905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc009b30 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0e9238640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000d800 tx=0x7ff0e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:57.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00096a0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0009800 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0009b70 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e41ef470 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e414f0d0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e404e170 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0e0009cd0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.287+0000 7ff0d9ffb640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 0x7ff0c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.291+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7ff0e00849b0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.291+0000 7ff0e8a37640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 0x7ff0c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.291+0000 7ff0e8a37640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 0x7ff0c8068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4150530 tx=0x7ff0d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.295+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0e0051b10 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.483+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0e40c82a0 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.483+0000 7ff0d9ffb640 1 -- 172.21.15.8:0/2739960905 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7ff0e0055130 con 0x7ff0e40cb8b0 2023-10-30T12:54:57.489 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:54:57.489 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:57.489 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:54:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 msgr2=0x7ff0c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 0x7ff0c8068960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4150530 tx=0x7ff0d4008040 comp rx=0 tx=0).stop 2023-10-30T12:54:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 msgr2=0x7ff0e41e3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000d800 tx=0x7ff0e000dcc0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 shutdown_connections 2023-10-30T12:54:57.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff0c80664a0 0x7ff0c8068960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0e4151c20 0x7ff0e4142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff0e4151840 0x7ff0e414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 --2- 172.21.15.8:0/2739960905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0e40cb8b0 0x7ff0e41e3770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 >> 172.21.15.8:0/2739960905 conn(0x7ff0e40cac70 msgr2=0x7ff0e4147a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 shutdown_connections 2023-10-30T12:54:57.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:57.487+0000 7ff0eb49c640 1 -- 172.21.15.8:0/2739960905 wait complete. 2023-10-30T12:54:57.510 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:54:59.511 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:54:59.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 Processor -- start 2023-10-30T12:54:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 -- start start 2023-10-30T12:54:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53b0153c80 0x7f53b015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f53b00c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b0159f20 con 0x7f53b015a3b0 2023-10-30T12:54:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b015dec0 con 0x7f53b0153c80 2023-10-30T12:54:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53b4817640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01599a0 con 0x7f53b00c4a10 2023-10-30T12:54:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f53b015a3b0 0x7f53b00ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40718/0 (socket says 172.21.15.8:40718) 2023-10-30T12:54:59.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 -- 172.21.15.8:0/163462554 learned_addr learned my addr 172.21.15.8:0/163462554 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:54:59.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53add9b640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f53b00c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ae59c640 1 --2- 172.21.15.8:0/163462554 >> v2:172.21.15.46:3300/0 conn(0x7f53b0153c80 0x7f53b015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287065414 0 0) 0x7f53b0159f20 con 0x7f53b015a3b0 2023-10-30T12:54:59.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5398003530 con 0x7f53b015a3b0 2023-10-30T12:54:59.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098536402 0 0) 0x7f5398003530 con 0x7f53b015a3b0 2023-10-30T12:54:59.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 msgr2=0x7f53b00c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:59.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f53b00c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 >> v2:172.21.15.46:3300/0 conn(0x7f53b0153c80 msgr2=0x7f53b015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:59.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 --2- 172.21.15.8:0/163462554 >> v2:172.21.15.46:3300/0 conn(0x7f53b0153c80 0x7f53b015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00c47e0 con 0x7f53b015a3b0 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f53a8001a00 con 0x7f53b015a3b0 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 >> v1:172.21.15.8:6789/0 conn(0x7f53b015a3b0 legacy=0x7f53b00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f5398003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398003f00 0x7f53980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5398004d10 0x7f5398005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015dec0 con 0x7f5398004d10 2023-10-30T12:54:59.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01599a0 con 0x7f5398003f00 2023-10-30T12:54:59.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539800ba70 con 0x7f53b00c4a10 2023-10-30T12:54:59.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ae59c640 1 --2- 172.21.15.8:0/163462554 >> v2:172.21.15.46:3300/0 conn(0x7f53b0153c80 0x7f53b015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:59.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5398004d10 0x7f5398005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53add9b640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398003f00 0x7f53980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53ae59c640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f5398003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 -- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 msgr2=0x7f5398003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:59.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f53b00c4a10 0x7f5398003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 -- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398003f00 msgr2=0x7f53980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:54:59.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398003f00 0x7f53980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53aed9d640 1 -- 172.21.15.8:0/163462554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398003530 con 0x7f5398004d10 2023-10-30T12:54:59.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.663+0000 7f53add9b640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398003f00 0x7f53980047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:54:59.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53aed9d640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5398004d10 0x7f5398005190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f53a800a860 tx=0x7f53a801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83cf92ed5258cb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80ad630 con 0x7f5398004d10 2023-10-30T12:54:59.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80a8070 con 0x7f5398004d10 2023-10-30T12:54:59.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53ad59a640 1 -- 172.21.15.8:0/163462554 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80ada30 con 0x7f5398004d10 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 -- 172.21.15.8:0/163462554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5398004d10 msgr2=0x7f5398005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 --2- 172.21.15.8:0/163462554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5398004d10 0x7f5398005190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f53a800a860 tx=0x7f53a801d360 comp rx=0 tx=0).stop 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53aed9d640 1 -- 172.21.15.8:0/163462554 reap_dead start 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 -- 172.21.15.8:0/163462554 shutdown_connections 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 -- 172.21.15.8:0/163462554 >> 172.21.15.8:0/163462554 conn(0x7f53b00715c0 msgr2=0x7f53b00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:59.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 -- 172.21.15.8:0/163462554 shutdown_connections 2023-10-30T12:54:59.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.667+0000 7f53b4817640 1 -- 172.21.15.8:0/163462554 wait complete. 2023-10-30T12:54:59.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 Processor -- start 2023-10-30T12:54:59.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- start start 2023-10-30T12:54:59.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5398004d10 0x7f53b00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53b00b5020 0x7f53b014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01ef440 con 0x7f53b0157b30 2023-10-30T12:54:59.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01ee8c0 con 0x7f53b00b5020 2023-10-30T12:54:59.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01eeba0 con 0x7f5398004d10 2023-10-30T12:54:59.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36876/0 (socket says 172.21.15.8:36876) 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 -- 172.21.15.8:0/2204150794 learned_addr learned my addr 172.21.15.8:0/2204150794 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53ae59c640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5398004d10 0x7f53b00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53aed9d640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53b00b5020 0x7f53b014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 -- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5398004d10 msgr2=0x7f53b00c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5398004d10 0x7f53b00c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 -- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53b00b5020 msgr2=0x7f53b014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53b00b5020 0x7f53b014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 -- 172.21.15.8:0/2204150794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01eee80 con 0x7f53b0157b30 2023-10-30T12:54:59.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53add9b640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006c10 tx=0x7f53a40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4004220 con 0x7f53b0157b30 2023-10-30T12:54:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a4004380 con 0x7f53b0157b30 2023-10-30T12:54:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4015870 con 0x7f53b0157b30 2023-10-30T12:54:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b01ef160 con 0x7f53b0157b30 2023-10-30T12:54:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.671+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b00c8e10 con 0x7f53b0157b30 2023-10-30T12:54:59.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b004e170 con 0x7f53b0157b30 2023-10-30T12:54:59.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53a40159d0 con 0x7f53b0157b30 2023-10-30T12:54:59.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f539effd640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 0x7f5390068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:54:59.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f53a40846d0 con 0x7f53b0157b30 2023-10-30T12:54:59.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f53ae59c640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 0x7f5390068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:54:59.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.675+0000 7f53ae59c640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 0x7f5390068990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53a0010840 tx=0x7f53a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:54:59.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.679+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53a4051830 con 0x7f53b0157b30 2023-10-30T12:54:59.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.871+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53b00cad40 con 0x7f53b0157b30 2023-10-30T12:54:59.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.871+0000 7f539effd640 1 -- 172.21.15.8:0/2204150794 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f53a4054e50 con 0x7f53b0157b30 2023-10-30T12:54:59.878 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:54:59.878 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:54:59.879 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:54:59.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 msgr2=0x7f5390068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 0x7f5390068990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53a0010840 tx=0x7f53a0008040 comp rx=0 tx=0).stop 2023-10-30T12:54:59.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 msgr2=0x7f53b014f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006c10 tx=0x7f53a40119d0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 shutdown_connections 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f53900664d0 0x7f5390068990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53b00b5020 0x7f53b014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53b0157b30 0x7f53b014f240 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 --2- 172.21.15.8:0/2204150794 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5398004d10 0x7f53b00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:54:59.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 >> 172.21.15.8:0/2204150794 conn(0x7f53b00715c0 msgr2=0x7f53b00c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:54:59.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 shutdown_connections 2023-10-30T12:54:59.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:54:59.875+0000 7f53b4817640 1 -- 172.21.15.8:0/2204150794 wait complete. 2023-10-30T12:54:59.899 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:01.901 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:02.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 Processor -- start 2023-10-30T12:55:02.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 -- start start 2023-10-30T12:55:02.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f529006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900c8440 con 0x7f529006a270 2023-10-30T12:55:02.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900c89f0 con 0x7f5290069e90 2023-10-30T12:55:02.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5294846640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5290153f60 con 0x7f52900c9c40 2023-10-30T12:55:02.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f528ed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f529006a270 0x7f52900c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40724/0 (socket says 172.21.15.8:40724) 2023-10-30T12:55:02.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f528ed9d640 1 -- 172.21.15.8:0/146609391 learned_addr learned my addr 172.21.15.8:0/146609391 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.051+0000 7f5287fff640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f529006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15854157 0 0) 0x7f52880014a0 con 0x7f529006a270 2023-10-30T12:55:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f527c003530 con 0x7f529006a270 2023-10-30T12:55:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875629267 0 0) 0x7f52900c89f0 con 0x7f5290069e90 2023-10-30T12:55:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52880014a0 con 0x7f5290069e90 2023-10-30T12:55:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744226243 0 0) 0x7f527c003530 con 0x7f529006a270 2023-10-30T12:55:02.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 msgr2=0x7f529006f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:02.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f529006f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 >> v1:172.21.15.46:6789/0 conn(0x7f5290069e90 legacy=0x7f52901597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900ca290 con 0x7f529006a270 2023-10-30T12:55:02.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f5288001b90 con 0x7f529006a270 2023-10-30T12:55:02.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 >> v1:172.21.15.8:6789/0 conn(0x7f529006a270 legacy=0x7f52900c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f527c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527c0047d0 0x7f527c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f527c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290153f60 con 0x7f527c003b40 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c00ba40 con 0x7f527c0047d0 2023-10-30T12:55:02.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c00bbd0 con 0x7f52900c9c40 2023-10-30T12:55:02.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f527c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528ed9d640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f527c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f5287fff640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527c0047d0 0x7f527c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 -- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 msgr2=0x7f527c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:02.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f52900c9c40 0x7f527c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 -- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527c0047d0 msgr2=0x7f527c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:02.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527c0047d0 0x7f527c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.055+0000 7f528e59c640 1 -- 172.21.15.8:0/146609391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c003530 con 0x7f527c003b40 2023-10-30T12:55:02.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f528e59c640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f527c003f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52800018d0 tx=0x7f528001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3d20c9bb733809 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800052d0 con 0x7f527c003b40 2023-10-30T12:55:02.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52800a7050 con 0x7f527c003b40 2023-10-30T12:55:02.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f528dd9b640 1 -- 172.21.15.8:0/146609391 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280003600 con 0x7f527c003b40 2023-10-30T12:55:02.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- 172.21.15.8:0/146609391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 msgr2=0x7f527c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 --2- 172.21.15.8:0/146609391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f527c003f40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52800018d0 tx=0x7f528001c500 comp rx=0 tx=0).stop 2023-10-30T12:55:02.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f528ed9d640 1 -- 172.21.15.8:0/146609391 reap_dead start 2023-10-30T12:55:02.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- 172.21.15.8:0/146609391 shutdown_connections 2023-10-30T12:55:02.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- 172.21.15.8:0/146609391 >> 172.21.15.8:0/146609391 conn(0x7f529006afa0 msgr2=0x7f529006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:02.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- 172.21.15.8:0/146609391 shutdown_connections 2023-10-30T12:55:02.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- 172.21.15.8:0/146609391 wait complete. 2023-10-30T12:55:02.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 Processor -- start 2023-10-30T12:55:02.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.059+0000 7f5294846640 1 -- start start 2023-10-30T12:55:02.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5290158ac0 0x7f52901632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5290158f40 0x7f5290163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52901511c0 con 0x7f527c003b40 2023-10-30T12:55:02.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290152010 con 0x7f5290158ac0 2023-10-30T12:55:02.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52901f5fa0 con 0x7f5290158f40 2023-10-30T12:55:02.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36900/0 (socket says 172.21.15.8:36900) 2023-10-30T12:55:02.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 -- 172.21.15.8:0/2889108529 learned_addr learned my addr 172.21.15.8:0/2889108529 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:02.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528ed9d640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5290158f40 0x7f5290163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5287fff640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5290158ac0 0x7f52901632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 -- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5290158f40 msgr2=0x7f5290163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5290158f40 0x7f5290163800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 -- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5290158ac0 msgr2=0x7f52901632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5290158ac0 0x7f52901632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 -- 172.21.15.8:0/2889108529 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290150d00 con 0x7f527c003b40 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f528e59c640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5280005040 tx=0x7f527c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280003ad0 con 0x7f527c003b40 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280003c30 con 0x7f527c003b40 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529014b5e0 con 0x7f527c003b40 2023-10-30T12:55:02.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800067e0 con 0x7f527c003b40 2023-10-30T12:55:02.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52901501a0 con 0x7f527c003b40 2023-10-30T12:55:02.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529015df90 con 0x7f527c003b40 2023-10-30T12:55:02.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.063+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5280006940 con 0x7f527c003b40 2023-10-30T12:55:02.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.067+0000 7f5286ffd640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 0x7f52700689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:02.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.067+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f528011dd60 con 0x7f527c003b40 2023-10-30T12:55:02.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.067+0000 7f5287fff640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 0x7f52700689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:02.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.067+0000 7f5287fff640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 0x7f52700689b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5278009a30 tx=0x7f5278008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.071+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f52800eea60 con 0x7f527c003b40 2023-10-30T12:55:02.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.263+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f529004e170 con 0x7f527c003b40 2023-10-30T12:55:02.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.263+0000 7f5286ffd640 1 -- 172.21.15.8:0/2889108529 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f52800bf090 con 0x7f527c003b40 2023-10-30T12:55:02.269 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:02.269 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:02.269 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:55:02.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 msgr2=0x7f52700689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 0x7f52700689b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5278009a30 tx=0x7f5278008040 comp rx=0 tx=0).stop 2023-10-30T12:55:02.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 msgr2=0x7f52901f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:02.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5280005040 tx=0x7f527c005800 comp rx=0 tx=0).stop 2023-10-30T12:55:02.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 shutdown_connections 2023-10-30T12:55:02.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52700664f0 0x7f52700689b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5290158ac0 0x7f52901632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5290158f40 0x7f5290163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 --2- 172.21.15.8:0/2889108529 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527c003b40 0x7f52901f5a60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:02.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 >> 172.21.15.8:0/2889108529 conn(0x7f529006afa0 msgr2=0x7f529015efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:02.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 shutdown_connections 2023-10-30T12:55:02.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:02.267+0000 7f5294846640 1 -- 172.21.15.8:0/2889108529 wait complete. 2023-10-30T12:55:02.290 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:04.291 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:04.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 Processor -- start 2023-10-30T12:55:04.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 -- start start 2023-10-30T12:55:04.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f24200c4a10 0x7f242015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f24200c42a0 0x7f24200ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2420159f20 con 0x7f2420153c80 2023-10-30T12:55:04.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242015dec0 con 0x7f24200c4a10 2023-10-30T12:55:04.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f242546b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24201599a0 con 0x7f24200c42a0 2023-10-30T12:55:04.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f241effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2420153c80 0x7f242015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:40740/0 (socket says 172.21.15.8:40740) 2023-10-30T12:55:04.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f241effd640 1 -- 172.21.15.8:0/1937935532 learned_addr learned my addr 172.21.15.8:0/1937935532 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:04.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f2417fff640 1 --2- 172.21.15.8:0/1937935532 >> v2:172.21.15.8:3300/0 conn(0x7f24200c4a10 0x7f242015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f241f7fe640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f24200c42a0 0x7f24200ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f241e7fc640 1 -- 172.21.15.8:0/1937935532 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436247424 0 0) 0x7f2420159f20 con 0x7f2420153c80 2023-10-30T12:55:04.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f2417fff640 1 -- 172.21.15.8:0/1937935532 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f24200c42a0 msgr2=0x7f24200ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:04.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f2417fff640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f24200c42a0 0x7f24200ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f2417fff640 1 -- 172.21.15.8:0/1937935532 >> v1:172.21.15.8:6789/0 conn(0x7f2420153c80 legacy=0x7f242015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.443+0000 7f2417fff640 1 -- 172.21.15.8:0/1937935532 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242015d620 con 0x7f24200c4a10 2023-10-30T12:55:04.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f2417fff640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c4a10 0x7f242015cf90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24080097c0 tx=0x7f240801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33386ede2f0c5e02 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:04.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f241e7fc640 1 -- 172.21.15.8:0/1937935532 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080a8970 con 0x7f24200c4a10 2023-10-30T12:55:04.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f241e7fc640 1 -- 172.21.15.8:0/1937935532 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24080a8ad0 con 0x7f24200c4a10 2023-10-30T12:55:04.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f241e7fc640 1 -- 172.21.15.8:0/1937935532 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080a8e60 con 0x7f24200c4a10 2023-10-30T12:55:04.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 -- 172.21.15.8:0/1937935532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c4a10 msgr2=0x7f242015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c4a10 0x7f242015cf90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24080097c0 tx=0x7f240801ec70 comp rx=0 tx=0).stop 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 -- 172.21.15.8:0/1937935532 shutdown_connections 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f24200c42a0 0x7f24200ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 --2- 172.21.15.8:0/1937935532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c4a10 0x7f242015cf90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 -- 172.21.15.8:0/1937935532 >> 172.21.15.8:0/1937935532 conn(0x7f24200715c0 msgr2=0x7f24200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 -- 172.21.15.8:0/1937935532 shutdown_connections 2023-10-30T12:55:04.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.447+0000 7f242546b640 1 -- 172.21.15.8:0/1937935532 wait complete. 2023-10-30T12:55:04.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 Processor -- start 2023-10-30T12:55:04.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 -- start start 2023-10-30T12:55:04.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24200c4a10 0x7f24200ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2420153c80 0x7f242015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242014b4a0 con 0x7f24200c42a0 2023-10-30T12:55:04.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242014b780 con 0x7f24200c4a10 2023-10-30T12:55:04.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f242546b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242014ba60 con 0x7f2420153c80 2023-10-30T12:55:04.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36922/0 (socket says 172.21.15.8:36922) 2023-10-30T12:55:04.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 -- 172.21.15.8:0/1109027544 learned_addr learned my addr 172.21.15.8:0/1109027544 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:04.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241f7fe640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2420153c80 0x7f242015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f2417fff640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24200c4a10 0x7f24200ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 -- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2420153c80 msgr2=0x7f242015c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2420153c80 0x7f242015c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 -- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24200c4a10 msgr2=0x7f24200ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24200c4a10 0x7f24200ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 -- 172.21.15.8:0/1109027544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408009420 con 0x7f24200c42a0 2023-10-30T12:55:04.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.451+0000 7f241effd640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f241000c480 tx=0x7f241000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:04.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100074e0 con 0x7f24200c42a0 2023-10-30T12:55:04.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2410007660 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410004a30 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242014bd40 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200ce410 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242004e170 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2410004b90 con 0x7f24200c42a0 2023-10-30T12:55:04.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 0x7f2400068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:04.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f2410083230 con 0x7f24200c42a0 2023-10-30T12:55:04.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.455+0000 7f2417fff640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 0x7f2400068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:04.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.459+0000 7f2417fff640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 0x7f2400068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f24080b2000 tx=0x7f2408008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.459+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2410053f40 con 0x7f24200c42a0 2023-10-30T12:55:04.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24200cad40 con 0x7f24200c42a0 2023-10-30T12:55:04.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f24177fe640 1 -- 172.21.15.8:0/1109027544 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f2410053ad0 con 0x7f24200c42a0 2023-10-30T12:55:04.660 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:04.660 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:04.660 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:55:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 msgr2=0x7f2400068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 0x7f2400068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f24080b2000 tx=0x7f2408008040 comp rx=0 tx=0).stop 2023-10-30T12:55:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 msgr2=0x7f242014a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f241000c480 tx=0x7f241000c940 comp rx=0 tx=0).stop 2023-10-30T12:55:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 shutdown_connections 2023-10-30T12:55:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f24000664a0 0x7f2400068960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24200c4a10 0x7f24200ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2420153c80 0x7f242015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 --2- 172.21.15.8:0/1109027544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24200c42a0 0x7f242014a9b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:04.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 >> 172.21.15.8:0/1109027544 conn(0x7f24200715c0 msgr2=0x7f24200cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:04.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 shutdown_connections 2023-10-30T12:55:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:04.655+0000 7f242546b640 1 -- 172.21.15.8:0/1109027544 wait complete. 2023-10-30T12:55:04.681 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:06.682 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:06.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.859+0000 7efc8759e640 1 Processor -- start 2023-10-30T12:55:06.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.863+0000 7efc8759e640 1 -- start start 2023-10-30T12:55:06.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.863+0000 7efc8759e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efc8814f830 0x7efc880c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.863+0000 7efc8759e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.863+0000 7efc8759e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc8806d4f0 0x7efc8806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc8759e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc880c6a70 con 0x7efc8814f830 2023-10-30T12:55:06.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc8759e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc881566c0 con 0x7efc8806d4f0 2023-10-30T12:55:06.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc8759e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc880c5b10 con 0x7efc8814fc10 2023-10-30T12:55:06.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:52144/0 (socket says 172.21.15.8:52144) 2023-10-30T12:55:06.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 -- 172.21.15.8:0/3056451777 learned_addr learned my addr 172.21.15.8:0/3056451777 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:06.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc86d9d640 1 --2- 172.21.15.8:0/3056451777 >> v2:172.21.15.46:3300/0 conn(0x7efc8806d4f0 0x7efc8806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 -- 172.21.15.8:0/3056451777 >> v2:172.21.15.46:3300/0 conn(0x7efc8806d4f0 msgr2=0x7efc8806d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:06.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 --2- 172.21.15.8:0/3056451777 >> v2:172.21.15.46:3300/0 conn(0x7efc8806d4f0 0x7efc8806d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 -- 172.21.15.8:0/3056451777 >> v2:172.21.15.8:3300/0 conn(0x7efc8814f830 msgr2=0x7efc880c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:06.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 --2- 172.21.15.8:0/3056451777 >> v2:172.21.15.8:3300/0 conn(0x7efc8814f830 0x7efc880c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.867+0000 7efc85d9b640 1 -- 172.21.15.8:0/3056451777 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc880c5df0 con 0x7efc8814fc10 2023-10-30T12:55:06.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.871+0000 7efc85d9b640 1 --2- 172.21.15.8:0/3056451777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc8806cc20 tx=0x7efc7801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed394e66e5ed5a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:06.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.875+0000 7efc8559a640 1 -- 172.21.15.8:0/3056451777 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780aa5f0 con 0x7efc8814fc10 2023-10-30T12:55:06.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.875+0000 7efc8559a640 1 -- 172.21.15.8:0/3056451777 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780aa750 con 0x7efc8814fc10 2023-10-30T12:55:06.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.875+0000 7efc8559a640 1 -- 172.21.15.8:0/3056451777 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780aaae0 con 0x7efc8814fc10 2023-10-30T12:55:06.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 -- 172.21.15.8:0/3056451777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 msgr2=0x7efc88156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:06.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 --2- 172.21.15.8:0/3056451777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc8806cc20 tx=0x7efc7801c510 comp rx=0 tx=0).stop 2023-10-30T12:55:06.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 -- 172.21.15.8:0/3056451777 shutdown_connections 2023-10-30T12:55:06.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 --2- 172.21.15.8:0/3056451777 >> v2:172.21.15.46:3300/0 conn(0x7efc8806d4f0 0x7efc8806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 --2- 172.21.15.8:0/3056451777 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88156180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 --2- 172.21.15.8:0/3056451777 >> v2:172.21.15.8:3300/0 conn(0x7efc8814f830 0x7efc880c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 -- 172.21.15.8:0/3056451777 >> 172.21.15.8:0/3056451777 conn(0x7efc880caae0 msgr2=0x7efc880caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:06.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 -- 172.21.15.8:0/3056451777 shutdown_connections 2023-10-30T12:55:06.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.879+0000 7efc8759e640 1 -- 172.21.15.8:0/3056451777 wait complete. 2023-10-30T12:55:06.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.883+0000 7efc8759e640 1 Processor -- start 2023-10-30T12:55:06.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.883+0000 7efc8759e640 1 -- start start 2023-10-30T12:55:06.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8806d4f0 0x7efc88155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 0x7efc88149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8814e380 con 0x7efc8814f830 2023-10-30T12:55:06.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc88153c90 con 0x7efc8806d4f0 2023-10-30T12:55:06.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8759e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc881536f0 con 0x7efc8814fc10 2023-10-30T12:55:06.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc85d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 0x7efc88149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc85d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 0x7efc88149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36952/0 (socket says 172.21.15.8:36952) 2023-10-30T12:55:06.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc85d9b640 1 -- 172.21.15.8:0/1002554636 learned_addr learned my addr 172.21.15.8:0/1002554636 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:06.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc8659c640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8806d4f0 0x7efc88155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 -- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8806d4f0 msgr2=0x7efc88155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:06.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8806d4f0 0x7efc88155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 -- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 msgr2=0x7efc88149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:06.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 0x7efc88149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:06.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.887+0000 7efc86d9d640 1 -- 172.21.15.8:0/1002554636 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780094d0 con 0x7efc8814fc10 2023-10-30T12:55:06.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc86d9d640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88149bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc80009d90 tx=0x7efc8000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:06.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80014480 con 0x7efc8814fc10 2023-10-30T12:55:06.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc881539b0 con 0x7efc8814fc10 2023-10-30T12:55:06.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8814dfa0 con 0x7efc8814fc10 2023-10-30T12:55:06.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8000f040 con 0x7efc8814fc10 2023-10-30T12:55:06.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.891+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800147e0 con 0x7efc8814fc10 2023-10-30T12:55:06.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.895+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6c005260 con 0x7efc8814fc10 2023-10-30T12:55:06.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.895+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc80014940 con 0x7efc8814fc10 2023-10-30T12:55:06.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.899+0000 7efc76ffd640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 0x7efc68068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:06.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.899+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7efc80085350 con 0x7efc8814fc10 2023-10-30T12:55:06.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.899+0000 7efc8659c640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 0x7efc68068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:06.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.899+0000 7efc8659c640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 0x7efc68068990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:06.931+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc80052670 con 0x7efc8814fc10 2023-10-30T12:55:07.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.127+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc6c0057e0 con 0x7efc8814fc10 2023-10-30T12:55:07.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc76ffd640 1 -- 172.21.15.8:0/1002554636 <== mon.2 v2:172.21.15.46:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7efc80055c90 con 0x7efc8814fc10 2023-10-30T12:55:07.135 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:07.135 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:07.135 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:55:07.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 msgr2=0x7efc68068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:07.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 0x7efc68068990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c008040 comp rx=0 tx=0).stop 2023-10-30T12:55:07.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 msgr2=0x7efc88149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:07.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88149bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc80009d90 tx=0x7efc8000c490 comp rx=0 tx=0).stop 2023-10-30T12:55:07.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 shutdown_connections 2023-10-30T12:55:07.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc680664d0 0x7efc68068990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:07.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc8814fc10 0x7efc88149bd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:07.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc8814f830 0x7efc88149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:07.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 --2- 172.21.15.8:0/1002554636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc8806d4f0 0x7efc88155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:07.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 >> 172.21.15.8:0/1002554636 conn(0x7efc880caae0 msgr2=0x7efc88159150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:07.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 shutdown_connections 2023-10-30T12:55:07.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:07.131+0000 7efc8759e640 1 -- 172.21.15.8:0/1002554636 wait complete. 2023-10-30T12:55:07.156 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:09.157 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:09.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.311+0000 7f1a4430e640 1 Processor -- start 2023-10-30T12:55:09.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.311+0000 7f1a4430e640 1 -- start start 2023-10-30T12:55:09.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a4430e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a3c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a4430e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c0c7b00 con 0x7f1a3c153aa0 2023-10-30T12:55:09.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a4430e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c06bd60 con 0x7f1a3c15d840 2023-10-30T12:55:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a4430e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c070a50 con 0x7f1a3c06b3b0 2023-10-30T12:55:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f1a3c153aa0 0x7f1a3c0701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50300/0 (socket says 172.21.15.8:50300) 2023-10-30T12:55:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 -- 172.21.15.8:0/867981053 learned_addr learned my addr 172.21.15.8:0/867981053 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:09.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a428ab640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a3c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905771412 0 0) 0x7f1a3c0c7b00 con 0x7f1a3c153aa0 2023-10-30T12:55:09.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a28003530 con 0x7f1a3c153aa0 2023-10-30T12:55:09.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2056805378 0 0) 0x7f1a28003530 con 0x7f1a3c153aa0 2023-10-30T12:55:09.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 msgr2=0x7f1a3c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:09.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a3c06b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 >> v1:172.21.15.46:6789/0 conn(0x7f1a3c15d840 legacy=0x7f1a3c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0c2050 con 0x7f1a3c153aa0 2023-10-30T12:55:09.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f1a38001680 con 0x7f1a3c153aa0 2023-10-30T12:55:09.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 >> v1:172.21.15.8:6789/0 conn(0x7f1a3c153aa0 legacy=0x7f1a3c0701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a28004360 0x7f1a28004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a28005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c070a50 con 0x7f1a280039b0 2023-10-30T12:55:09.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2800b990 con 0x7f1a28004360 2023-10-30T12:55:09.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2800bb50 con 0x7f1a3c06b3b0 2023-10-30T12:55:09.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a418a9640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a28004360 0x7f1a28004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a428ab640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a28005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 -- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 msgr2=0x7f1a28005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:09.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c06b3b0 0x7f1a28005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 -- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a28004360 msgr2=0x7f1a28004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:09.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a28004360 0x7f1a28004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.315+0000 7f1a420aa640 1 -- 172.21.15.8:0/867981053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28003530 con 0x7f1a280039b0 2023-10-30T12:55:09.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a420aa640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a28003d90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800c210 tx=0x7f1a3801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b4a06c80503781 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:09.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380a5b60 con 0x7f1a280039b0 2023-10-30T12:55:09.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a380a5ce0 con 0x7f1a280039b0 2023-10-30T12:55:09.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a410a8640 1 -- 172.21.15.8:0/867981053 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38005b20 con 0x7f1a280039b0 2023-10-30T12:55:09.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 -- 172.21.15.8:0/867981053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 msgr2=0x7f1a28003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 --2- 172.21.15.8:0/867981053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a28003d90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800c210 tx=0x7f1a3801cc60 comp rx=0 tx=0).stop 2023-10-30T12:55:09.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a428ab640 1 -- 172.21.15.8:0/867981053 reap_dead start 2023-10-30T12:55:09.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 -- 172.21.15.8:0/867981053 shutdown_connections 2023-10-30T12:55:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 -- 172.21.15.8:0/867981053 >> 172.21.15.8:0/867981053 conn(0x7f1a3c071fb0 msgr2=0x7f1a3c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 -- 172.21.15.8:0/867981053 shutdown_connections 2023-10-30T12:55:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.319+0000 7f1a4430e640 1 -- 172.21.15.8:0/867981053 wait complete. 2023-10-30T12:55:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 Processor -- start 2023-10-30T12:55:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- start start 2023-10-30T12:55:09.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a3c1ef760 0x7f1a3c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c0c54f0 0x7f1a3c0c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c48e0 con 0x7f1a280039b0 2023-10-30T12:55:09.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c1f2270 con 0x7f1a3c1ef760 2023-10-30T12:55:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c1f3240 con 0x7f1a3c0c54f0 2023-10-30T12:55:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37742/0 (socket says 172.21.15.8:37742) 2023-10-30T12:55:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 -- 172.21.15.8:0/834039755 learned_addr learned my addr 172.21.15.8:0/834039755 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a418a9640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a3c1ef760 0x7f1a3c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a428ab640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c0c54f0 0x7f1a3c0c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 -- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c0c54f0 msgr2=0x7f1a3c0c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c0c54f0 0x7f1a3c0c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 -- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a3c1ef760 msgr2=0x7f1a3c14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a3c1ef760 0x7f1a3c14f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 -- 172.21.15.8:0/834039755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c1f2db0 con 0x7f1a280039b0 2023-10-30T12:55:09.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a420aa640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3801c6d0 tx=0x7f1a28005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:09.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380b4070 con 0x7f1a280039b0 2023-10-30T12:55:09.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a380a5b70 con 0x7f1a280039b0 2023-10-30T12:55:09.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380049f0 con 0x7f1a280039b0 2023-10-30T12:55:09.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c0cca90 con 0x7f1a280039b0 2023-10-30T12:55:09.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.323+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c1f2760 con 0x7f1a280039b0 2023-10-30T12:55:09.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c04e170 con 0x7f1a280039b0 2023-10-30T12:55:09.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1a38004b50 con 0x7f1a280039b0 2023-10-30T12:55:09.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a32ffd640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 0x7f1a20068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:09.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f1a3811dbc0 con 0x7f1a280039b0 2023-10-30T12:55:09.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a418a9640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 0x7f1a20068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:09.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.327+0000 7f1a418a9640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 0x7f1a20068990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15a920 tx=0x7f1a2c00a860 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:09.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.331+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1a380eaef0 con 0x7f1a280039b0 2023-10-30T12:55:09.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.519+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a3c14e540 con 0x7f1a280039b0 2023-10-30T12:55:09.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.519+0000 7f1a32ffd640 1 -- 172.21.15.8:0/834039755 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f1a380ee510 con 0x7f1a280039b0 2023-10-30T12:55:09.526 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:09.526 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:09.527 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:55:09.527 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 msgr2=0x7f1a20068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 0x7f1a20068990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15a920 tx=0x7f1a2c00a860 comp rx=0 tx=0).stop 2023-10-30T12:55:09.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 msgr2=0x7f1a3c0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:09.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3801c6d0 tx=0x7f1a28005800 comp rx=0 tx=0).stop 2023-10-30T12:55:09.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 shutdown_connections 2023-10-30T12:55:09.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1a200664d0 0x7f1a20068990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a3c1ef760 0x7f1a3c14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1a3c0c54f0 0x7f1a3c0c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 --2- 172.21.15.8:0/834039755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a280039b0 0x7f1a3c0cc450 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:09.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 >> 172.21.15.8:0/834039755 conn(0x7f1a3c071fb0 msgr2=0x7f1a3c0733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:09.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 shutdown_connections 2023-10-30T12:55:09.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:09.523+0000 7f1a4430e640 1 -- 172.21.15.8:0/834039755 wait complete. 2023-10-30T12:55:09.547 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:11.548 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:11.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 Processor -- start 2023-10-30T12:55:11.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 -- start start 2023-10-30T12:55:11.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7c60153c80 0x7f7c6015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c600c4a10 0x7f7c600c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c60159f20 con 0x7f7c6015a3b0 2023-10-30T12:55:11.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6015dec0 con 0x7f7c60153c80 2023-10-30T12:55:11.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.691+0000 7f7c64c68640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c601599a0 con 0x7f7c600c4a10 2023-10-30T12:55:11.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5ed9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7c6015a3b0 0x7f7c600ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50308/0 (socket says 172.21.15.8:50308) 2023-10-30T12:55:11.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7c60153c80 0x7f7c6015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5ed9d640 1 -- 172.21.15.8:0/457002290 learned_addr learned my addr 172.21.15.8:0/457002290 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:11.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5dd9b640 1 --2- 172.21.15.8:0/457002290 >> v2:172.21.15.46:3300/0 conn(0x7f7c600c4a10 0x7f7c600c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 -- 172.21.15.8:0/457002290 >> v2:172.21.15.46:3300/0 conn(0x7f7c600c4a10 msgr2=0x7f7c600c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:11.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 --2- 172.21.15.8:0/457002290 >> v2:172.21.15.46:3300/0 conn(0x7f7c600c4a10 0x7f7c600c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 -- 172.21.15.8:0/457002290 >> v1:172.21.15.8:6789/0 conn(0x7f7c6015a3b0 legacy=0x7f7c600ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 -- 172.21.15.8:0/457002290 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600c47e0 con 0x7f7c60153c80 2023-10-30T12:55:11.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.695+0000 7f7c5e59c640 1 --2- 172.21.15.8:0/457002290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c60153c80 0x7f7c6015cf90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c6006be50 tx=0x7f7c5001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeaa153c06475e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:11.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c5d59a640 1 -- 172.21.15.8:0/457002290 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500a8950 con 0x7f7c60153c80 2023-10-30T12:55:11.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c5d59a640 1 -- 172.21.15.8:0/457002290 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500a8ab0 con 0x7f7c60153c80 2023-10-30T12:55:11.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c5d59a640 1 -- 172.21.15.8:0/457002290 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500a8e40 con 0x7f7c60153c80 2023-10-30T12:55:11.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- 172.21.15.8:0/457002290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c60153c80 msgr2=0x7f7c6015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 --2- 172.21.15.8:0/457002290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c60153c80 0x7f7c6015cf90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c6006be50 tx=0x7f7c5001ec50 comp rx=0 tx=0).stop 2023-10-30T12:55:11.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- 172.21.15.8:0/457002290 shutdown_connections 2023-10-30T12:55:11.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 --2- 172.21.15.8:0/457002290 >> v2:172.21.15.46:3300/0 conn(0x7f7c600c4a10 0x7f7c600c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 --2- 172.21.15.8:0/457002290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c60153c80 0x7f7c6015cf90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- 172.21.15.8:0/457002290 >> 172.21.15.8:0/457002290 conn(0x7f7c600715c0 msgr2=0x7f7c600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:11.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- 172.21.15.8:0/457002290 shutdown_connections 2023-10-30T12:55:11.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- 172.21.15.8:0/457002290 wait complete. 2023-10-30T12:55:11.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 Processor -- start 2023-10-30T12:55:11.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.699+0000 7f7c64c68640 1 -- start start 2023-10-30T12:55:11.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c60153c80 0x7f7c601f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7c6015a3b0 0x7f7c60145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c601531f0 con 0x7f7c600c4a10 2023-10-30T12:55:11.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6015ab60 con 0x7f7c60153c80 2023-10-30T12:55:11.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6015ae00 con 0x7f7c6015a3b0 2023-10-30T12:55:11.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37760/0 (socket says 172.21.15.8:37760) 2023-10-30T12:55:11.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 -- 172.21.15.8:0/1672610830 learned_addr learned my addr 172.21.15.8:0/1672610830 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:11.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5dd9b640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c60153c80 0x7f7c601f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5ed9d640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7c6015a3b0 0x7f7c60145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 -- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7c6015a3b0 msgr2=0x7f7c60145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7c6015a3b0 0x7f7c60145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 -- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c60153c80 msgr2=0x7f7c601f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c60153c80 0x7f7c601f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 -- 172.21.15.8:0/1672610830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009420 con 0x7f7c600c4a10 2023-10-30T12:55:11.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c5e59c640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b3040 tx=0x7f7c5001b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:11.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500b0df0 con 0x7f7c600c4a10 2023-10-30T12:55:11.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500b9410 con 0x7f7c600c4a10 2023-10-30T12:55:11.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500b97c0 con 0x7f7c600c4a10 2023-10-30T12:55:11.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c601f2cb0 con 0x7f7c600c4a10 2023-10-30T12:55:11.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6015c820 con 0x7f7c600c4a10 2023-10-30T12:55:11.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c601529d0 con 0x7f7c600c4a10 2023-10-30T12:55:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.703+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c500b7040 con 0x7f7c600c4a10 2023-10-30T12:55:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.707+0000 7f7c4effd640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 0x7f7c40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.707+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f7c50121b80 con 0x7f7c600c4a10 2023-10-30T12:55:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.707+0000 7f7c5dd9b640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 0x7f7c40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.707+0000 7f7c5dd9b640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 0x7f7c40068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7c60149ca0 tx=0x7f7c54009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:11.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.711+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c500eee20 con 0x7f7c600c4a10 2023-10-30T12:55:11.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c6004e170 con 0x7f7c600c4a10 2023-10-30T12:55:11.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c4effd640 1 -- 172.21.15.8:0/1672610830 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f7c500b5090 con 0x7f7c600c4a10 2023-10-30T12:55:11.908 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:11.908 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:54:39.277056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":4929},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4929":{"gid":4929,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":33,"addr":"172.21.15.8:6839/3916953811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":3916953811},{"type":"v1","addr":"172.21.15.8:6839","nonce":3916953811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:11.909 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 51 2023-10-30T12:55:11.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 msgr2=0x7f7c40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 0x7f7c40068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7c60149ca0 tx=0x7f7c54009190 comp rx=0 tx=0).stop 2023-10-30T12:55:11.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 msgr2=0x7f7c60152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:11.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b3040 tx=0x7f7c5001b9f0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 shutdown_connections 2023-10-30T12:55:11.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7c400664a0 0x7f7c40068960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c60153c80 0x7f7c601f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7c6015a3b0 0x7f7c60145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 --2- 172.21.15.8:0/1672610830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c600c4a10 0x7f7c60152cb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:11.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 >> 172.21.15.8:0/1672610830 conn(0x7f7c600715c0 msgr2=0x7f7c600caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:11.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 shutdown_connections 2023-10-30T12:55:11.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:11.903+0000 7f7c64c68640 1 -- 172.21.15.8:0/1672610830 wait complete. 2023-10-30T12:55:11.930 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-30T12:55:13.931 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:14.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.079+0000 7fecba5d3640 1 Processor -- start 2023-10-30T12:55:14.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.079+0000 7fecba5d3640 1 -- start start 2023-10-30T12:55:14.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecba5d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb41516c0 0x7fecb40c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecba5d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fecb40c2500 0x7fecb40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecba5d3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb4151aa0 con 0x7fecb40c82d0 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecba5d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb40c74a0 con 0x7fecb40c2500 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecba5d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40c2e90 con 0x7fecb41516c0 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fecb40c2500 0x7fecb40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fecb40c2500 0x7fecb40c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37762/0 (socket says 172.21.15.8:37762) 2023-10-30T12:55:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 -- 172.21.15.8:0/4075122357 learned_addr learned my addr 172.21.15.8:0/4075122357 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb3fff640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb41516c0 0x7fecb40c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 -- 172.21.15.8:0/4075122357 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb41516c0 msgr2=0x7fecb40c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb41516c0 0x7fecb40c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 -- 172.21.15.8:0/4075122357 >> v1:172.21.15.46:6789/0 conn(0x7fecb40c82d0 legacy=0x7fecb40ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.083+0000 7fecb8b70640 1 -- 172.21.15.8:0/4075122357 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4158af0 con 0x7fecb40c2500 2023-10-30T12:55:14.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecb8b70640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb40c2500 0x7fecb40c2950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fecac009e80 tx=0x7fecac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257ec7f8b70524b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecb2ffd640 1 -- 172.21.15.8:0/4075122357 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0a56c0 con 0x7fecb40c2500 2023-10-30T12:55:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecb2ffd640 1 -- 172.21.15.8:0/4075122357 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0a5820 con 0x7fecb40c2500 2023-10-30T12:55:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecb2ffd640 1 -- 172.21.15.8:0/4075122357 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0a5bb0 con 0x7fecb40c2500 2023-10-30T12:55:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- 172.21.15.8:0/4075122357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb40c2500 msgr2=0x7fecb40c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb40c2500 0x7fecb40c2950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fecac009e80 tx=0x7fecac01c9f0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- 172.21.15.8:0/4075122357 shutdown_connections 2023-10-30T12:55:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb40c2500 0x7fecb40c2950 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 --2- 172.21.15.8:0/4075122357 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb41516c0 0x7fecb40c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- 172.21.15.8:0/4075122357 >> 172.21.15.8:0/4075122357 conn(0x7fecb4071fb0 msgr2=0x7fecb40723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:14.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- 172.21.15.8:0/4075122357 shutdown_connections 2023-10-30T12:55:14.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- 172.21.15.8:0/4075122357 wait complete. 2023-10-30T12:55:14.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 Processor -- start 2023-10-30T12:55:14.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.087+0000 7fecba5d3640 1 -- start start 2023-10-30T12:55:14.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb40c2500 0x7fecb414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40c82d0 0x7fecb4160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4141340 con 0x7fecb41516c0 2023-10-30T12:55:14.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb413f700 con 0x7fecb40c82d0 2023-10-30T12:55:14.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4140280 con 0x7fecb40c2500 2023-10-30T12:55:14.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37776/0 (socket says 172.21.15.8:37776) 2023-10-30T12:55:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 -- 172.21.15.8:0/1122590489 learned_addr learned my addr 172.21.15.8:0/1122590489 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb3fff640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb40c2500 0x7fecb414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb37fe640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40c82d0 0x7fecb4160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 -- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb40c2500 msgr2=0x7fecb414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb40c2500 0x7fecb414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 -- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40c82d0 msgr2=0x7fecb4160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40c82d0 0x7fecb4160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 -- 172.21.15.8:0/1122590489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009b30 con 0x7fecb41516c0 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb8b70640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecac01b0a0 tx=0x7fecac019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0a56c0 con 0x7fecb41516c0 2023-10-30T12:55:14.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0b88a0 con 0x7fecb41516c0 2023-10-30T12:55:14.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0b8c50 con 0x7fecb41516c0 2023-10-30T12:55:14.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb4160620 con 0x7fecb41516c0 2023-10-30T12:55:14.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb4140d80 con 0x7fecb41516c0 2023-10-30T12:55:14.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb41607b0 con 0x7fecb41516c0 2023-10-30T12:55:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.091+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fecac0c3460 con 0x7fecb41516c0 2023-10-30T12:55:14.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.095+0000 7fecb0ff9640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 0x7feca0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.095+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fecac11f6c0 con 0x7fecb41516c0 2023-10-30T12:55:14.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.095+0000 7fecb3fff640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 0x7feca0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.095+0000 7fecb3fff640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 0x7feca0068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feca4007a60 tx=0x7feca4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.099+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fecac0ec9a0 con 0x7fecb41516c0 2023-10-30T12:55:14.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecb404e170 con 0x7fecb41516c0 2023-10-30T12:55:14.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecb0ff9640 1 -- 172.21.15.8:0/1122590489 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fecac0effc0 con 0x7fecb41516c0 2023-10-30T12:55:14.292 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:14.293 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:12.201837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:14.293 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 52 2023-10-30T12:55:14.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 msgr2=0x7feca0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 0x7feca0068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feca4007a60 tx=0x7feca4008040 comp rx=0 tx=0).stop 2023-10-30T12:55:14.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 msgr2=0x7fecb4160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.287+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecac01b0a0 tx=0x7fecac019790 comp rx=0 tx=0).stop 2023-10-30T12:55:14.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 shutdown_connections 2023-10-30T12:55:14.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7feca00664a0 0x7feca0068960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40c82d0 0x7fecb4160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fecb41516c0 0x7fecb4160ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 --2- 172.21.15.8:0/1122590489 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fecb40c2500 0x7fecb414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 >> 172.21.15.8:0/1122590489 conn(0x7fecb4071fb0 msgr2=0x7fecb4072390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:14.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 shutdown_connections 2023-10-30T12:55:14.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.291+0000 7fecba5d3640 1 -- 172.21.15.8:0/1122590489 wait complete. 2023-10-30T12:55:14.313 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-30T12:55:14.314 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:55:14.314 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:14.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 Processor -- start 2023-10-30T12:55:14.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 -- start start 2023-10-30T12:55:14.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1d78150830 0x7f1d7806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d780709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d780c2310 con 0x7f1d780cb7d0 2023-10-30T12:55:14.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d78156710 con 0x7f1d78150830 2023-10-30T12:55:14.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.451+0000 7f1d7f580640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780c4360 con 0x7f1d78070520 2023-10-30T12:55:14.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1d78150830 0x7f1d7806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1d78150830 0x7f1d7806b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37782/0 (socket says 172.21.15.8:37782) 2023-10-30T12:55:14.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 -- 172.21.15.8:0/3942233032 learned_addr learned my addr 172.21.15.8:0/3942233032 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:14.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7db1d640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d780709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 -- 172.21.15.8:0/3942233032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 msgr2=0x7f1d780709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:14.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d780709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 -- 172.21.15.8:0/3942233032 >> v1:172.21.15.46:6789/0 conn(0x7f1d780cb7d0 legacy=0x7f1d780c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 -- 172.21.15.8:0/3942233032 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d781569b0 con 0x7f1d78150830 2023-10-30T12:55:14.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d7d31c640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d78150830 0x7f1d7806b500 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009870 tx=0x7f1d74020f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3502c893c2c75c server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d6ffff640 1 -- 172.21.15.8:0/3942233032 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d740aac40 con 0x7f1d78150830 2023-10-30T12:55:14.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d6ffff640 1 -- 172.21.15.8:0/3942233032 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d740aada0 con 0x7f1d78150830 2023-10-30T12:55:14.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.455+0000 7f1d6ffff640 1 -- 172.21.15.8:0/3942233032 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d740b2660 con 0x7f1d78150830 2023-10-30T12:55:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- 172.21.15.8:0/3942233032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d78150830 msgr2=0x7f1d7806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d78150830 0x7f1d7806b500 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009870 tx=0x7f1d74020f40 comp rx=0 tx=0).stop 2023-10-30T12:55:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- 172.21.15.8:0/3942233032 shutdown_connections 2023-10-30T12:55:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d780709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- 172.21.15.8:0/3942233032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d78150830 0x7f1d7806b500 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- 172.21.15.8:0/3942233032 >> 172.21.15.8:0/3942233032 conn(0x7f1d780cac70 msgr2=0x7f1d780cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:14.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- 172.21.15.8:0/3942233032 shutdown_connections 2023-10-30T12:55:14.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- 172.21.15.8:0/3942233032 wait complete. 2023-10-30T12:55:14.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 Processor -- start 2023-10-30T12:55:14.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- start start 2023-10-30T12:55:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d781512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 0x7f1d781e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78152500 con 0x7f1d780cb7d0 2023-10-30T12:55:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78152da0 con 0x7f1d78150830 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7f580640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d781e1680 con 0x7f1d78070520 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7cb1b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7cb1b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37786/0 (socket says 172.21.15.8:37786) 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7cb1b640 1 -- 172.21.15.8:0/1507815146 learned_addr learned my addr 172.21.15.8:0/1507815146 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7d31c640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d781512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.459+0000 7f1d7db1d640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 0x7f1d781e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 -- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 msgr2=0x7f1d781512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d781512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 -- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 msgr2=0x7f1d781e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 0x7f1d781e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 -- 172.21.15.8:0/1507815146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740094d0 con 0x7f1d780cb7d0 2023-10-30T12:55:14.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7db1d640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 0x7f1d781e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:14.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7cb1b640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800c450 tx=0x7f1d6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d680052d0 con 0x7f1d780cb7d0 2023-10-30T12:55:14.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d68005430 con 0x7f1d780cb7d0 2023-10-30T12:55:14.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68007260 con 0x7f1d780cb7d0 2023-10-30T12:55:14.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d78151950 con 0x7f1d780cb7d0 2023-10-30T12:55:14.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d78152750 con 0x7f1d780cb7d0 2023-10-30T12:55:14.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7804e170 con 0x7f1d780cb7d0 2023-10-30T12:55:14.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d680073c0 con 0x7f1d780cb7d0 2023-10-30T12:55:14.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 0x7f1d5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:14.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f1d680838e0 con 0x7f1d780cb7d0 2023-10-30T12:55:14.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.463+0000 7f1d7d31c640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 0x7f1d5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:14.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.467+0000 7f1d7d31c640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 0x7f1d5c068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d740b4000 tx=0x7f1d740ac070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:14.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.467+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d68050b10 con 0x7f1d780cb7d0 2023-10-30T12:55:14.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.659+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d78070300 con 0x7f1d780cb7d0 2023-10-30T12:55:14.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.659+0000 7f1d6dffb640 1 -- 172.21.15.8:0/1507815146 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f1d68054130 con 0x7f1d780cb7d0 2023-10-30T12:55:14.664 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:14.664 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:12.201837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:14.665 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 52 2023-10-30T12:55:14.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 msgr2=0x7f1d5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 0x7f1d5c068960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d740b4000 tx=0x7f1d740ac070 comp rx=0 tx=0).stop 2023-10-30T12:55:14.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 msgr2=0x7f1d781e1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:14.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800c450 tx=0x7f1d6800c910 comp rx=0 tx=0).stop 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 shutdown_connections 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1d5c0664a0 0x7f1d5c068960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d78150830 0x7f1d781e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d780cb7d0 0x7f1d781e1140 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 --2- 172.21.15.8:0/1507815146 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1d78070520 0x7f1d781512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:14.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 >> 172.21.15.8:0/1507815146 conn(0x7f1d780cac70 msgr2=0x7f1d780c6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:14.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 shutdown_connections 2023-10-30T12:55:14.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:14.663+0000 7f1d7f580640 1 -- 172.21.15.8:0/1507815146 wait complete. 2023-10-30T12:55:14.686 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 52, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 52, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 26 }, "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 31 }, "gid_5394": { "addr": "172.21.15.8:6837/1849048959", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 1849048959, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 1849048959, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5394, "incarnation": 52, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:55:12.201837+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5079, "mds_2": 5394 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:55:15.672 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:15.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:16.687 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:16.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 Processor -- start 2023-10-30T12:55:16.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 -- start start 2023-10-30T12:55:16.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb830071340 0x7fb830065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830159970 0x7fb830159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83006e390 con 0x7fb830153c10 2023-10-30T12:55:16.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb830072390 con 0x7fb830071340 2023-10-30T12:55:16.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb836009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300660e0 con 0x7fb830159970 2023-10-30T12:55:16.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82f7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fb830153c10 0x7fb830070c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50320/0 (socket says 172.21.15.8:50320) 2023-10-30T12:55:16.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82effd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb830071340 0x7fb830065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82f7fe640 1 -- 172.21.15.8:0/1380844710 learned_addr learned my addr 172.21.15.8:0/1380844710 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:16.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82ffff640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830159970 0x7fb830159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82effd640 1 -- 172.21.15.8:0/1380844710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830159970 msgr2=0x7fb830159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:16.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82effd640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830159970 0x7fb830159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82effd640 1 -- 172.21.15.8:0/1380844710 >> v1:172.21.15.8:6789/0 conn(0x7fb830153c10 legacy=0x7fb830070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:16.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.839+0000 7fb82effd640 1 -- 172.21.15.8:0/1380844710 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300bec10 con 0x7fb830071340 2023-10-30T12:55:16.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb82effd640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830071340 0x7fb830065ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097c0 tx=0x7fb82401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a953570517a17152 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:16.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb82e7fc640 1 -- 172.21.15.8:0/1380844710 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8240ab600 con 0x7fb830071340 2023-10-30T12:55:16.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb82e7fc640 1 -- 172.21.15.8:0/1380844710 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8240a7050 con 0x7fb830071340 2023-10-30T12:55:16.849 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb82e7fc640 1 -- 172.21.15.8:0/1380844710 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8240ab980 con 0x7fb830071340 2023-10-30T12:55:16.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 -- 172.21.15.8:0/1380844710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830071340 msgr2=0x7fb830065ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:16.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830071340 0x7fb830065ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097c0 tx=0x7fb82401c3f0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 -- 172.21.15.8:0/1380844710 shutdown_connections 2023-10-30T12:55:16.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830159970 0x7fb830159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.850 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 --2- 172.21.15.8:0/1380844710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830071340 0x7fb830065ba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 -- 172.21.15.8:0/1380844710 >> 172.21.15.8:0/1380844710 conn(0x7fb8300c7f60 msgr2=0x7fb8300c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:16.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 -- 172.21.15.8:0/1380844710 shutdown_connections 2023-10-30T12:55:16.851 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.843+0000 7fb836009640 1 -- 172.21.15.8:0/1380844710 wait complete. 2023-10-30T12:55:16.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 Processor -- start 2023-10-30T12:55:16.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- start start 2023-10-30T12:55:16.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb830071340 0x7fb83014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830153c10 0x7fb8300c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300ca520 con 0x7fb830159970 2023-10-30T12:55:16.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830146950 con 0x7fb830071340 2023-10-30T12:55:16.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83014e0d0 con 0x7fb830153c10 2023-10-30T12:55:16.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37810/0 (socket says 172.21.15.8:37810) 2023-10-30T12:55:16.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 -- 172.21.15.8:0/2111334271 learned_addr learned my addr 172.21.15.8:0/2111334271 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:16.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82f7fe640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb830071340 0x7fb83014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82effd640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830153c10 0x7fb8300c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 -- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830153c10 msgr2=0x7fb8300c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830153c10 0x7fb8300c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 -- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb830071340 msgr2=0x7fb83014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:16.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb830071340 0x7fb83014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:16.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 -- 172.21.15.8:0/2111334271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb824009420 con 0x7fb830159970 2023-10-30T12:55:16.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb82ffff640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb828007e70 tx=0x7fb82800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:16.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828014620 con 0x7fb830159970 2023-10-30T12:55:16.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8280147a0 con 0x7fb830159970 2023-10-30T12:55:16.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828014b50 con 0x7fb830159970 2023-10-30T12:55:16.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb830149b30 con 0x7fb830159970 2023-10-30T12:55:16.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.847+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8300c92c0 con 0x7fb830159970 2023-10-30T12:55:16.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83004e170 con 0x7fb830159970 2023-10-30T12:55:16.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb828014cb0 con 0x7fb830159970 2023-10-30T12:55:16.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb817fff640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 0x7fb81c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:16.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fb828084b60 con 0x7fb830159970 2023-10-30T12:55:16.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb82f7fe640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 0x7fb81c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:16.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.851+0000 7fb82f7fe640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 0x7fb81c068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb820005890 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:16.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:16.855+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb828051c70 con 0x7fb830159970 2023-10-30T12:55:17.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.047+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb830158220 con 0x7fb830159970 2023-10-30T12:55:17.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.047+0000 7fb817fff640 1 -- 172.21.15.8:0/2111334271 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fb828055290 con 0x7fb830159970 2023-10-30T12:55:17.052 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:17.052 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:12.201837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:17.052 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 52 2023-10-30T12:55:17.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 msgr2=0x7fb81c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 0x7fb81c068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb820005890 tx=0x7fb820009040 comp rx=0 tx=0).stop 2023-10-30T12:55:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 msgr2=0x7fb8300c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb828007e70 tx=0x7fb82800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:55:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 shutdown_connections 2023-10-30T12:55:17.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb81c0664d0 0x7fb81c068990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:17.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb830159970 0x7fb8300c9fe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:17.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb830153c10 0x7fb8300c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:17.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 --2- 172.21.15.8:0/2111334271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb830071340 0x7fb83014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:17.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 >> 172.21.15.8:0/2111334271 conn(0x7fb8300c7f60 msgr2=0x7fb83006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:17.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 shutdown_connections 2023-10-30T12:55:17.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:17.051+0000 7fb836009640 1 -- 172.21.15.8:0/2111334271 wait complete. 2023-10-30T12:55:17.108 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:19.110 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:19.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 Processor -- start 2023-10-30T12:55:19.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 -- start start 2023-10-30T12:55:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f526015cf90 0x7f52600c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5260159f20 con 0x7f52600c4a10 2023-10-30T12:55:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600c47e0 con 0x7f5260153c80 2023-10-30T12:55:19.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f5265dd3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526015dec0 con 0x7f526015cf90 2023-10-30T12:55:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525ffff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f526015cf90 0x7f52600c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f52600c4a10 0x7f52600ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38742/0 (socket says 172.21.15.8:38742) 2023-10-30T12:55:19.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525effd640 1 -- 172.21.15.8:0/2535056493 learned_addr learned my addr 172.21.15.8:0/2535056493 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525ffff640 1 -- 172.21.15.8:0/2535056493 >> v1:172.21.15.46:6789/0 conn(0x7f5260153c80 legacy=0x7f526015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525ffff640 1 -- 172.21.15.8:0/2535056493 >> v1:172.21.15.8:6789/0 conn(0x7f52600c4a10 legacy=0x7f52600ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525ffff640 1 -- 172.21.15.8:0/2535056493 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526015d620 con 0x7f526015cf90 2023-10-30T12:55:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.259+0000 7f525e7fc640 1 -- 172.21.15.8:0/2535056493 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908592318 0 0) 0x7f5260159f20 con 0x7f52600c4a10 2023-10-30T12:55:19.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f525ffff640 1 --2- 172.21.15.8:0/2535056493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f526015cf90 0x7f52600c42a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f312dfb0ecce5d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f525e7fc640 1 -- 172.21.15.8:0/2535056493 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52580af600 con 0x7f526015cf90 2023-10-30T12:55:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f525e7fc640 1 -- 172.21.15.8:0/2535056493 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52580ab050 con 0x7f526015cf90 2023-10-30T12:55:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f525e7fc640 1 -- 172.21.15.8:0/2535056493 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52580af980 con 0x7f526015cf90 2023-10-30T12:55:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 -- 172.21.15.8:0/2535056493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f526015cf90 msgr2=0x7f52600c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 --2- 172.21.15.8:0/2535056493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f526015cf90 0x7f52600c42a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525801f130 comp rx=0 tx=0).stop 2023-10-30T12:55:19.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 -- 172.21.15.8:0/2535056493 shutdown_connections 2023-10-30T12:55:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 --2- 172.21.15.8:0/2535056493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f526015cf90 0x7f52600c42a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 -- 172.21.15.8:0/2535056493 >> 172.21.15.8:0/2535056493 conn(0x7f52600715c0 msgr2=0x7f52600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 -- 172.21.15.8:0/2535056493 shutdown_connections 2023-10-30T12:55:19.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.263+0000 7f5265dd3640 1 -- 172.21.15.8:0/2535056493 wait complete. 2023-10-30T12:55:19.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 Processor -- start 2023-10-30T12:55:19.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 -- start start 2023-10-30T12:55:19.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260153c80 0x7f5260148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f526015cf90 0x7f52601eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52601efa50 con 0x7f52600c4a10 2023-10-30T12:55:19.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52600b2020 con 0x7f5260153c80 2023-10-30T12:55:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52600b4df0 con 0x7f526015cf90 2023-10-30T12:55:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39628/0 (socket says 172.21.15.8:39628) 2023-10-30T12:55:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 -- 172.21.15.8:0/4099675195 learned_addr learned my addr 172.21.15.8:0/4099675195 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:19.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525ffff640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f526015cf90 0x7f52601eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525effd640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260153c80 0x7f5260148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 -- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f526015cf90 msgr2=0x7f52601eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f526015cf90 0x7f52601eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 -- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260153c80 msgr2=0x7f5260148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260153c80 0x7f5260148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 -- 172.21.15.8:0/4099675195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258009a70 con 0x7f52600c4a10 2023-10-30T12:55:19.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f525f7fe640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f525000c500 tx=0x7f525000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250005220 con 0x7f52600c4a10 2023-10-30T12:55:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250005380 con 0x7f52600c4a10 2023-10-30T12:55:19.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250007270 con 0x7f52600c4a10 2023-10-30T12:55:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.267+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52600c5cc0 con 0x7f52600c4a10 2023-10-30T12:55:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52600b2270 con 0x7f52600c4a10 2023-10-30T12:55:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526004e170 con 0x7f52600c4a10 2023-10-30T12:55:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f52500073d0 con 0x7f52600c4a10 2023-10-30T12:55:19.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f5247fff640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 0x7f524c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f52500834e0 con 0x7f52600c4a10 2023-10-30T12:55:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f525effd640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 0x7f524c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:19.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.271+0000 7f525effd640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 0x7f524c068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5254007fd0 tx=0x7f5254007ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:19.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.275+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5250050690 con 0x7f52600c4a10 2023-10-30T12:55:19.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52600cad40 con 0x7f52600c4a10 2023-10-30T12:55:19.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5247fff640 1 -- 172.21.15.8:0/4099675195 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f5250053cb0 con 0x7f52600c4a10 2023-10-30T12:55:19.472 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:19.473 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:17.832231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:resolve","state_seq":29,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:19.473 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 53 2023-10-30T12:55:19.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 msgr2=0x7f524c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 0x7f524c068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5254007fd0 tx=0x7f5254007ba0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 msgr2=0x7f52600b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.467+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f525000c500 tx=0x7f525000c9c0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 shutdown_connections 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f524c0664a0 0x7f524c068960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260153c80 0x7f5260148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f526015cf90 0x7f52601eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 --2- 172.21.15.8:0/4099675195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52600c4a10 0x7f52600b3410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:19.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 >> 172.21.15.8:0/4099675195 conn(0x7f52600715c0 msgr2=0x7f5260151640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:19.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 shutdown_connections 2023-10-30T12:55:19.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:19.471+0000 7f5265dd3640 1 -- 172.21.15.8:0/4099675195 wait complete. 2023-10-30T12:55:19.493 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:21.495 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:21.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 Processor -- start 2023-10-30T12:55:21.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 -- start start 2023-10-30T12:55:21.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 0x7f4c540c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c54151aa0 con 0x7f4c540c82d0 2023-10-30T12:55:21.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540c74a0 con 0x7f4c540c2500 2023-10-30T12:55:21.653 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c5b93e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540c2e90 con 0x7f4c541516c0 2023-10-30T12:55:21.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c59edb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 0x7f4c540c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c59edb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 0x7f4c540c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:44620/0 (socket says 172.21.15.8:44620) 2023-10-30T12:55:21.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:46368/0 (socket says 172.21.15.8:46368) 2023-10-30T12:55:21.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c59edb640 1 -- 172.21.15.8:0/2269741350 learned_addr learned my addr 172.21.15.8:0/2269741350 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:21.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c58ed9640 1 --1- 172.21.15.8:0/2269741350 >> v1:172.21.15.46:6789/0 conn(0x7f4c540c82d0 0x7f4c540ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:48694/0 (socket says 172.21.15.8:48694) 2023-10-30T12:55:21.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c4bfff640 1 -- 172.21.15.8:0/2269741350 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635523350 0 0) 0x7f4c54151aa0 con 0x7f4c540c82d0 2023-10-30T12:55:21.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c4bfff640 1 -- 172.21.15.8:0/2269741350 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c40003530 con 0x7f4c540c82d0 2023-10-30T12:55:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 -- 172.21.15.8:0/2269741350 >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 msgr2=0x7f4c540c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 --2- 172.21.15.8:0/2269741350 >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 0x7f4c540c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 -- 172.21.15.8:0/2269741350 >> v1:172.21.15.46:6789/0 conn(0x7f4c540c82d0 legacy=0x7f4c540ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.647+0000 7f4c596da640 1 -- 172.21.15.8:0/2269741350 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54158af0 con 0x7f4c541516c0 2023-10-30T12:55:21.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c596da640 1 --2- 172.21.15.8:0/2269741350 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097c0 tx=0x7f4c5001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b03ac327057e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c4bfff640 1 -- 172.21.15.8:0/2269741350 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500ab600 con 0x7f4c541516c0 2023-10-30T12:55:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c4bfff640 1 -- 172.21.15.8:0/2269741350 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500a7050 con 0x7f4c541516c0 2023-10-30T12:55:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c4bfff640 1 -- 172.21.15.8:0/2269741350 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500ab980 con 0x7f4c541516c0 2023-10-30T12:55:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 -- 172.21.15.8:0/2269741350 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 msgr2=0x7f4c540c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/2269741350 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097c0 tx=0x7f4c5001c480 comp rx=0 tx=0).stop 2023-10-30T12:55:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 -- 172.21.15.8:0/2269741350 shutdown_connections 2023-10-30T12:55:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/2269741350 >> v2:172.21.15.46:3300/0 conn(0x7f4c540c2500 0x7f4c540c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/2269741350 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c541516c0 0x7f4c540c7d90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 -- 172.21.15.8:0/2269741350 >> 172.21.15.8:0/2269741350 conn(0x7f4c54071fb0 msgr2=0x7f4c540723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 -- 172.21.15.8:0/2269741350 shutdown_connections 2023-10-30T12:55:21.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.651+0000 7f4c5b93e640 1 -- 172.21.15.8:0/2269741350 wait complete. 2023-10-30T12:55:21.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 Processor -- start 2023-10-30T12:55:21.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- start start 2023-10-30T12:55:21.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 0x7f4c540c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 0x7f4c54145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541f4fe0 con 0x7f4c540c2500 2023-10-30T12:55:21.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541f2970 con 0x7f4c541516c0 2023-10-30T12:55:21.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540c95c0 con 0x7f4c540c82d0 2023-10-30T12:55:21.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39630/0 (socket says 172.21.15.8:39630) 2023-10-30T12:55:21.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 -- 172.21.15.8:0/4218056836 learned_addr learned my addr 172.21.15.8:0/4218056836 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:21.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c59edb640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 0x7f4c54145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c58ed9640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 0x7f4c540c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 -- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 msgr2=0x7f4c540c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 0x7f4c540c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 -- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 msgr2=0x7f4c54145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 0x7f4c54145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 -- 172.21.15.8:0/4218056836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50009420 con 0x7f4c540c2500 2023-10-30T12:55:21.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c58ed9640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 0x7f4c540c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:21.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c59edb640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 0x7f4c54145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:21.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c596da640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c500ae040 tx=0x7f4c50019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:21.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500b4610 con 0x7f4c540c2500 2023-10-30T12:55:21.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500b4770 con 0x7f4c540c2500 2023-10-30T12:55:21.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500b4b00 con 0x7f4c540c2500 2023-10-30T12:55:21.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c541f5210 con 0x7f4c540c2500 2023-10-30T12:55:21.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.655+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c541f2040 con 0x7f4c540c2500 2023-10-30T12:55:21.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5404e170 con 0x7f4c540c2500 2023-10-30T12:55:21.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4c500b2040 con 0x7f4c540c2500 2023-10-30T12:55:21.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c49ffb640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 0x7f4c38068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:21.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f4c5011e230 con 0x7f4c540c2500 2023-10-30T12:55:21.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c58ed9640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 0x7f4c38068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:21.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.659+0000 7f4c58ed9640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 0x7f4c38068990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4c54072a70 tx=0x7f4c44007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:21.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.663+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4c500eb340 con 0x7f4c540c2500 2023-10-30T12:55:21.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c54155250 con 0x7f4c540c2500 2023-10-30T12:55:21.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c49ffb640 1 -- 172.21.15.8:0/4218056836 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f4c500b0020 con 0x7f4c540c2500 2023-10-30T12:55:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:21.859 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:17.832231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:resolve","state_seq":29,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:21.860 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 53 2023-10-30T12:55:21.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 msgr2=0x7f4c38068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 0x7f4c38068990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4c54072a70 tx=0x7f4c44007c40 comp rx=0 tx=0).stop 2023-10-30T12:55:21.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 msgr2=0x7f4c540cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:21.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c500ae040 tx=0x7f4c50019220 comp rx=0 tx=0).stop 2023-10-30T12:55:21.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 shutdown_connections 2023-10-30T12:55:21.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4c380664d0 0x7f4c38068990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c541516c0 0x7f4c54145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4c540c82d0 0x7f4c540c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 --2- 172.21.15.8:0/4218056836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c540c2500 0x7f4c540cb1a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:21.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 >> 172.21.15.8:0/4218056836 conn(0x7f4c54071fb0 msgr2=0x7f4c540c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:21.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 shutdown_connections 2023-10-30T12:55:21.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:21.855+0000 7f4c5b93e640 1 -- 172.21.15.8:0/4218056836 wait complete. 2023-10-30T12:55:21.881 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:23.882 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:24.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.035+0000 7fda71854640 1 Processor -- start 2023-10-30T12:55:24.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.035+0000 7fda71854640 1 -- start start 2023-10-30T12:55:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c153c80 0x7fda6c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda6c0c4a10 0x7fda6c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda6c15a3b0 0x7fda6c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c15dec0 con 0x7fda6c0c4a10 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c1599a0 con 0x7fda6c15a3b0 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda71854640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c159f20 con 0x7fda6c153c80 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda6c0c4a10 0x7fda6c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fda6c0c4a10 0x7fda6c0c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39644/0 (socket says 172.21.15.8:39644) 2023-10-30T12:55:24.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 -- 172.21.15.8:0/3673208391 learned_addr learned my addr 172.21.15.8:0/3673208391 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6affd640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c153c80 0x7fda6c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6b7fe640 1 --2- 172.21.15.8:0/3673208391 >> v2:172.21.15.46:3300/0 conn(0x7fda6c15a3b0 0x7fda6c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 -- 172.21.15.8:0/3673208391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c153c80 msgr2=0x7fda6c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:24.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c153c80 0x7fda6c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 -- 172.21.15.8:0/3673208391 >> v2:172.21.15.46:3300/0 conn(0x7fda6c15a3b0 msgr2=0x7fda6c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:24.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/3673208391 >> v2:172.21.15.46:3300/0 conn(0x7fda6c15a3b0 0x7fda6c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.039+0000 7fda6a7fc640 1 -- 172.21.15.8:0/3673208391 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0c47e0 con 0x7fda6c0c4a10 2023-10-30T12:55:24.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c0c4a10 0x7fda6c0c42a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0cbbc0 tx=0x7fda6001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84570a60cbaf050 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:24.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda69ffb640 1 -- 172.21.15.8:0/3673208391 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600a89b0 con 0x7fda6c0c4a10 2023-10-30T12:55:24.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda69ffb640 1 -- 172.21.15.8:0/3673208391 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600a8b10 con 0x7fda6c0c4a10 2023-10-30T12:55:24.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda69ffb640 1 -- 172.21.15.8:0/3673208391 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600a8ea0 con 0x7fda6c0c4a10 2023-10-30T12:55:24.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- 172.21.15.8:0/3673208391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c0c4a10 msgr2=0x7fda6c0c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:24.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c0c4a10 0x7fda6c0c42a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0cbbc0 tx=0x7fda6001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.048 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- 172.21.15.8:0/3673208391 shutdown_connections 2023-10-30T12:55:24.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 --2- 172.21.15.8:0/3673208391 >> v2:172.21.15.46:3300/0 conn(0x7fda6c15a3b0 0x7fda6c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c0c4a10 0x7fda6c0c42a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 --2- 172.21.15.8:0/3673208391 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c153c80 0x7fda6c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- 172.21.15.8:0/3673208391 >> 172.21.15.8:0/3673208391 conn(0x7fda6c0715c0 msgr2=0x7fda6c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:24.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- 172.21.15.8:0/3673208391 shutdown_connections 2023-10-30T12:55:24.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- 172.21.15.8:0/3673208391 wait complete. 2023-10-30T12:55:24.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 Processor -- start 2023-10-30T12:55:24.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.043+0000 7fda71854640 1 -- start start 2023-10-30T12:55:24.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 0x7fda6c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c15a3b0 0x7fda6c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14df30 con 0x7fda6c153c80 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14e480 con 0x7fda6c15a3b0 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c14f820 con 0x7fda6c0c4a10 2023-10-30T12:55:24.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39646/0 (socket says 172.21.15.8:39646) 2023-10-30T12:55:24.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 -- 172.21.15.8:0/1511379072 learned_addr learned my addr 172.21.15.8:0/1511379072 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:24.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6affd640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 0x7fda6c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6b7fe640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c15a3b0 0x7fda6c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 -- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 msgr2=0x7fda6c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:24.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 0x7fda6c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 -- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c15a3b0 msgr2=0x7fda6c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:24.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c15a3b0 0x7fda6c0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 -- 172.21.15.8:0/1511379072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600094d0 con 0x7fda6c153c80 2023-10-30T12:55:24.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6affd640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 0x7fda6c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:24.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda6a7fc640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda6001d3a0 tx=0x7fda6001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:24.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600b1db0 con 0x7fda6c153c80 2023-10-30T12:55:24.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600b9440 con 0x7fda6c153c80 2023-10-30T12:55:24.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600b97d0 con 0x7fda6c153c80 2023-10-30T12:55:24.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c14ee50 con 0x7fda6c153c80 2023-10-30T12:55:24.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c14fd40 con 0x7fda6c153c80 2023-10-30T12:55:24.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c04e170 con 0x7fda6c153c80 2023-10-30T12:55:24.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.047+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda600b7070 con 0x7fda6c153c80 2023-10-30T12:55:24.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.051+0000 7fda53fff640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 0x7fda58068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:24.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.051+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fda60121b70 con 0x7fda6c153c80 2023-10-30T12:55:24.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.051+0000 7fda6affd640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 0x7fda58068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:24.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.051+0000 7fda6affd640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 0x7fda58068990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda5c004660 tx=0x7fda5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:24.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.055+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda600eed10 con 0x7fda6c153c80 2023-10-30T12:55:24.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda6c0cb9d0 con 0x7fda6c153c80 2023-10-30T12:55:24.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda53fff640 1 -- 172.21.15.8:0/1511379072 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5735 (secure 0 0 0) 0x7fda600b5020 con 0x7fda6c153c80 2023-10-30T12:55:24.248 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:24.249 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:23.242987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:24.249 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 54 2023-10-30T12:55:24.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 msgr2=0x7fda58068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:24.250 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 0x7fda58068990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda5c004660 tx=0x7fda5c008040 comp rx=0 tx=0).stop 2023-10-30T12:55:24.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 msgr2=0x7fda6c0ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:24.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.243+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda6001d3a0 tx=0x7fda6001ba90 comp rx=0 tx=0).stop 2023-10-30T12:55:24.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 shutdown_connections 2023-10-30T12:55:24.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fda580664d0 0x7fda58068990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c15a3b0 0x7fda6c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda6c153c80 0x7fda6c0ce890 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 --2- 172.21.15.8:0/1511379072 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fda6c0c4a10 0x7fda6c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:24.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 >> 172.21.15.8:0/1511379072 conn(0x7fda6c0715c0 msgr2=0x7fda6c158470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:24.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 shutdown_connections 2023-10-30T12:55:24.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:24.247+0000 7fda71854640 1 -- 172.21.15.8:0/1511379072 wait complete. 2023-10-30T12:55:24.269 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:26.271 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:26.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.427+0000 7fe92933e640 1 Processor -- start 2023-10-30T12:55:26.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.427+0000 7fe92933e640 1 -- start start 2023-10-30T12:55:26.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.427+0000 7fe92933e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe924153c80 0x7fe92415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe92933e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9240c42a0 0x7fe9240ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe92933e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe924159f20 con 0x7fe9240c4a10 2023-10-30T12:55:26.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe92933e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92415dec0 con 0x7fe924153c80 2023-10-30T12:55:26.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe92933e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9241599a0 con 0x7fe9240c42a0 2023-10-30T12:55:26.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe924153c80 0x7fe92415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:26.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe91bfff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe9240c4a10 0x7fe92415a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38758/0 (socket says 172.21.15.8:38758) 2023-10-30T12:55:26.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe91bfff640 1 -- 172.21.15.8:0/1990589760 learned_addr learned my addr 172.21.15.8:0/1990589760 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:26.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe9237fe640 1 --2- 172.21.15.8:0/1990589760 >> v2:172.21.15.46:3300/0 conn(0x7fe9240c42a0 0x7fe9240ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:26.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe9227fc640 1 -- 172.21.15.8:0/1990589760 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771567955 0 0) 0x7fe924159f20 con 0x7fe9240c4a10 2023-10-30T12:55:26.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe9227fc640 1 -- 172.21.15.8:0/1990589760 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003530 con 0x7fe9240c4a10 2023-10-30T12:55:26.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 -- 172.21.15.8:0/1990589760 >> v2:172.21.15.46:3300/0 conn(0x7fe9240c42a0 msgr2=0x7fe9240ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:26.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 --2- 172.21.15.8:0/1990589760 >> v2:172.21.15.46:3300/0 conn(0x7fe9240c42a0 0x7fe9240ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 -- 172.21.15.8:0/1990589760 >> v1:172.21.15.8:6789/0 conn(0x7fe9240c4a10 legacy=0x7fe92415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 -- 172.21.15.8:0/1990589760 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92415d620 con 0x7fe924153c80 2023-10-30T12:55:26.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.431+0000 7fe922ffd640 1 --2- 172.21.15.8:0/1990589760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe924153c80 0x7fe92415cf90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b1d40 tx=0x7fe91401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b23cfdb0bb98092 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:26.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe9227fc640 1 -- 172.21.15.8:0/1990589760 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140a8950 con 0x7fe924153c80 2023-10-30T12:55:26.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe9227fc640 1 -- 172.21.15.8:0/1990589760 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9140a8ab0 con 0x7fe924153c80 2023-10-30T12:55:26.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe9227fc640 1 -- 172.21.15.8:0/1990589760 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140a8e40 con 0x7fe924153c80 2023-10-30T12:55:26.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- 172.21.15.8:0/1990589760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe924153c80 msgr2=0x7fe92415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- 172.21.15.8:0/1990589760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe924153c80 0x7fe92415cf90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b1d40 tx=0x7fe91401ec50 comp rx=0 tx=0).stop 2023-10-30T12:55:26.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- 172.21.15.8:0/1990589760 shutdown_connections 2023-10-30T12:55:26.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- 172.21.15.8:0/1990589760 >> v2:172.21.15.46:3300/0 conn(0x7fe9240c42a0 0x7fe9240ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- 172.21.15.8:0/1990589760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe924153c80 0x7fe92415cf90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- 172.21.15.8:0/1990589760 >> 172.21.15.8:0/1990589760 conn(0x7fe9240715c0 msgr2=0x7fe9240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- 172.21.15.8:0/1990589760 shutdown_connections 2023-10-30T12:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- 172.21.15.8:0/1990589760 wait complete. 2023-10-30T12:55:26.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 Processor -- start 2023-10-30T12:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- start start 2023-10-30T12:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 0x7fe9240c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe924153c80 0x7fe9240b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240b3d10 con 0x7fe9240c42a0 2023-10-30T12:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240b3380 con 0x7fe9240c4a10 2023-10-30T12:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe92933e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240b3700 con 0x7fe924153c80 2023-10-30T12:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe922ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe922ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39660/0 (socket says 172.21.15.8:39660) 2023-10-30T12:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe922ffd640 1 -- 172.21.15.8:0/873769538 learned_addr learned my addr 172.21.15.8:0/873769538 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe91bfff640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 0x7fe9240c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe922ffd640 1 -- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe924153c80 msgr2=0x7fe9240b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.435+0000 7fe922ffd640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe924153c80 0x7fe9240b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe922ffd640 1 -- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 msgr2=0x7fe9240c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe922ffd640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 0x7fe9240c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe922ffd640 1 -- 172.21.15.8:0/873769538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009420 con 0x7fe9240c42a0 2023-10-30T12:55:26.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91bfff640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 0x7fe9240c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:55:26.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe922ffd640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b3040 tx=0x7fe91401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:26.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b0df0 con 0x7fe9240c42a0 2023-10-30T12:55:26.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9140b9410 con 0x7fe9240c42a0 2023-10-30T12:55:26.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b97a0 con 0x7fe9240c42a0 2023-10-30T12:55:26.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92415c650 con 0x7fe9240c42a0 2023-10-30T12:55:26.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9240b39a0 con 0x7fe9240c42a0 2023-10-30T12:55:26.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92404e170 con 0x7fe9240c42a0 2023-10-30T12:55:26.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9140b7040 con 0x7fe9240c42a0 2023-10-30T12:55:26.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 0x7fe9040689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:26.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fe914121ac0 con 0x7fe9240c42a0 2023-10-30T12:55:26.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.439+0000 7fe91bfff640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 0x7fe9040689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:26.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.443+0000 7fe91bfff640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 0x7fe9040689b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b5320 tx=0x7fe90c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:26.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.443+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9140eedf0 con 0x7fe9240c42a0 2023-10-30T12:55:26.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.635+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9240cad40 con 0x7fe9240c42a0 2023-10-30T12:55:26.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.635+0000 7fe91b7fe640 1 -- 172.21.15.8:0/873769538 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5735 (secure 0 0 0) 0x7fe9140b5090 con 0x7fe9240c42a0 2023-10-30T12:55:26.641 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:26.641 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:23.242987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 54 2023-10-30T12:55:26.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 msgr2=0x7fe9040689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 0x7fe9040689b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b5320 tx=0x7fe90c009040 comp rx=0 tx=0).stop 2023-10-30T12:55:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 msgr2=0x7fe92415b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b3040 tx=0x7fe91401b9f0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 shutdown_connections 2023-10-30T12:55:26.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe9040664f0 0x7fe9040689b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9240c4a10 0x7fe9240c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe924153c80 0x7fe9240b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 --2- 172.21.15.8:0/873769538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9240c42a0 0x7fe92415b570 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 >> 172.21.15.8:0/873769538 conn(0x7fe9240715c0 msgr2=0x7fe92406bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:26.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 shutdown_connections 2023-10-30T12:55:26.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:26.639+0000 7fe92933e640 1 -- 172.21.15.8:0/873769538 wait complete. 2023-10-30T12:55:26.662 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 54, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5079": { "addr": "172.21.15.8:6838/3033347051", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6835", "nonce": 3033347051, "type": "v2" }, { "addr": "172.21.15.8:6838", "nonce": 3033347051, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5079, "incarnation": 31, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 26 }, "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 31 }, "gid_5394": { "addr": "172.21.15.8:6837/1849048959", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 1849048959, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 1849048959, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5394, "incarnation": 52, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:55:23.242987+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5079, "mds_2": 5394 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:55:28.665 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 Processor -- start 2023-10-30T12:55:28.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 -- start start 2023-10-30T12:55:28.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 0x7f76b415e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 0x7f76b40c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b4071e00 con 0x7f76b406bb40 2023-10-30T12:55:28.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b4070020 con 0x7f76b406c420 2023-10-30T12:55:28.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.815+0000 7f76bbd1d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b415eb90 con 0x7f76b406b760 2023-10-30T12:55:28.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b92b8640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f76b406bb40 0x7f76b4157a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38768/0 (socket says 172.21.15.8:38768) 2023-10-30T12:55:28.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b92b8640 1 -- 172.21.15.8:0/740415782 learned_addr learned my addr 172.21.15.8:0/740415782 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:28.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/740415782 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 0x7f76b415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76ba2ba640 1 --2- 172.21.15.8:0/740415782 >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 0x7f76b40c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b8ab7640 1 -- 172.21.15.8:0/740415782 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306980868 0 0) 0x7f76b4071e00 con 0x7f76b406bb40 2023-10-30T12:55:28.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b8ab7640 1 -- 172.21.15.8:0/740415782 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a0003530 con 0x7f76b406bb40 2023-10-30T12:55:28.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 -- 172.21.15.8:0/740415782 >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 msgr2=0x7f76b40c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:28.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/740415782 >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 0x7f76b40c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:28.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 -- 172.21.15.8:0/740415782 >> v1:172.21.15.8:6789/0 conn(0x7f76b406bb40 legacy=0x7f76b4157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:28.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 -- 172.21.15.8:0/740415782 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b4070f10 con 0x7f76b406b760 2023-10-30T12:55:28.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76ba2ba640 1 --2- 172.21.15.8:0/740415782 >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 0x7f76b40c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:55:28.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.819+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/740415782 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 0x7f76b415e650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76b00097c0 tx=0x7f76b001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c19ea78a2b5f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:28.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76b8ab7640 1 -- 172.21.15.8:0/740415782 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00ab650 con 0x7f76b406b760 2023-10-30T12:55:28.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76b8ab7640 1 -- 172.21.15.8:0/740415782 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00ab7b0 con 0x7f76b406b760 2023-10-30T12:55:28.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76b8ab7640 1 -- 172.21.15.8:0/740415782 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00abb20 con 0x7f76b406b760 2023-10-30T12:55:28.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- 172.21.15.8:0/740415782 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 msgr2=0x7f76b415e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:28.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/740415782 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 0x7f76b415e650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76b00097c0 tx=0x7f76b001c450 comp rx=0 tx=0).stop 2023-10-30T12:55:28.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- 172.21.15.8:0/740415782 shutdown_connections 2023-10-30T12:55:28.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/740415782 >> v2:172.21.15.46:3300/0 conn(0x7f76b406c420 0x7f76b40c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:28.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/740415782 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406b760 0x7f76b415e650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:28.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- 172.21.15.8:0/740415782 >> 172.21.15.8:0/740415782 conn(0x7f76b40cac70 msgr2=0x7f76b40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:28.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- 172.21.15.8:0/740415782 shutdown_connections 2023-10-30T12:55:28.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- 172.21.15.8:0/740415782 wait complete. 2023-10-30T12:55:28.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 Processor -- start 2023-10-30T12:55:28.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- start start 2023-10-30T12:55:28.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b406bb40 0x7f76b40c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406c420 0x7f76b40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b415a1e0 con 0x7f76b406b760 2023-10-30T12:55:28.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b415a4a0 con 0x7f76b406bb40 2023-10-30T12:55:28.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76bbd1d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4158750 con 0x7f76b406c420 2023-10-30T12:55:28.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.823+0000 7f76b9ab9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39672/0 (socket says 172.21.15.8:39672) 2023-10-30T12:55:28.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 -- 172.21.15.8:0/11138658 learned_addr learned my addr 172.21.15.8:0/11138658 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:28.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b92b8640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b406bb40 0x7f76b40c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76ba2ba640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406c420 0x7f76b40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 -- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406c420 msgr2=0x7f76b40c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:28.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406c420 0x7f76b40c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:28.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 -- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b406bb40 msgr2=0x7f76b40c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:28.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b406bb40 0x7f76b40c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:28.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 -- 172.21.15.8:0/11138658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009a70 con 0x7f76b406b760 2023-10-30T12:55:28.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76b9ab9640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f76b001ccf0 tx=0x7f76b001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:28.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00bf610 con 0x7f76b406b760 2023-10-30T12:55:28.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00bf770 con 0x7f76b406b760 2023-10-30T12:55:28.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00bfae0 con 0x7f76b406b760 2023-10-30T12:55:28.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b0009420 con 0x7f76b406b760 2023-10-30T12:55:28.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b40ca9f0 con 0x7f76b406b760 2023-10-30T12:55:28.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b404e170 con 0x7f76b406b760 2023-10-30T12:55:28.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f76b00bfc40 con 0x7f76b406b760 2023-10-30T12:55:28.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.827+0000 7f76aa7fc640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 0x7f7698068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:28.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.831+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f76b011efc0 con 0x7f76b406b760 2023-10-30T12:55:28.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.831+0000 7f76b92b8640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 0x7f7698068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:28.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.831+0000 7f76b92b8640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 0x7f7698068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f76b40c3680 tx=0x7f76a4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:28.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:28.835+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f76b00ec1f0 con 0x7f76b406b760 2023-10-30T12:55:29.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76b4066510 con 0x7f76b406b760 2023-10-30T12:55:29.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76aa7fc640 1 -- 172.21.15.8:0/11138658 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f76b00ef810 con 0x7f76b406b760 2023-10-30T12:55:29.028 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:29.028 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:23.242987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:29.029 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 54 2023-10-30T12:55:29.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 msgr2=0x7f7698068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:29.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 0x7f7698068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f76b40c3680 tx=0x7f76a4009040 comp rx=0 tx=0).stop 2023-10-30T12:55:29.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 msgr2=0x7f76b40c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:29.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.023+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f76b001ccf0 tx=0x7f76b001ac90 comp rx=0 tx=0).stop 2023-10-30T12:55:29.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 shutdown_connections 2023-10-30T12:55:29.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f76980664a0 0x7f7698068960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:29.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b406bb40 0x7f76b40c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:29.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f76b406c420 0x7f76b40c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:29.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 --2- 172.21.15.8:0/11138658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f76b406b760 0x7f76b40c2d50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:29.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 >> 172.21.15.8:0/11138658 conn(0x7f76b40cac70 msgr2=0x7f76b4154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:29.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 shutdown_connections 2023-10-30T12:55:29.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:29.027+0000 7f76bbd1d640 1 -- 172.21.15.8:0/11138658 wait complete. 2023-10-30T12:55:29.049 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:31.050 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:31.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 Processor -- start 2023-10-30T12:55:31.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 -- start start 2023-10-30T12:55:31.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb438150830 0x7fb43806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4380709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4380c2310 con 0x7fb4380cb7d0 2023-10-30T12:55:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb438156710 con 0x7fb438150830 2023-10-30T12:55:31.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43fc32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c4360 con 0x7fb438070520 2023-10-30T12:55:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43d1cd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fb4380cb7d0 0x7fb4380c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43978/0 (socket says 172.21.15.8:43978) 2023-10-30T12:55:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.199+0000 7fb43d1cd640 1 -- 172.21.15.8:0/3003019843 learned_addr learned my addr 172.21.15.8:0/3003019843 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4380709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.207 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/3003019843 >> v2:172.21.15.46:3300/0 conn(0x7fb438150830 0x7fb43806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752303084 0 0) 0x7fb4380c2310 con 0x7fb4380cb7d0 2023-10-30T12:55:31.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb424003530 con 0x7fb4380cb7d0 2023-10-30T12:55:31.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175533155 0 0) 0x7fb424003530 con 0x7fb4380cb7d0 2023-10-30T12:55:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 msgr2=0x7fb4380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4380709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 >> v2:172.21.15.46:3300/0 conn(0x7fb438150830 msgr2=0x7fb43806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:31.210 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 --2- 172.21.15.8:0/3003019843 >> v2:172.21.15.46:3300/0 conn(0x7fb438150830 0x7fb43806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4381569b0 con 0x7fb4380cb7d0 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fb428001680 con 0x7fb4380cb7d0 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 >> v1:172.21.15.8:6789/0 conn(0x7fb4380cb7d0 legacy=0x7fb4380c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb424003980 0x7fb424003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb424004d10 0x7fb424005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438156710 con 0x7fb424004d10 2023-10-30T12:55:31.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c4360 con 0x7fb424003980 2023-10-30T12:55:31.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42400b990 con 0x7fb438070520 2023-10-30T12:55:31.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb424004d10 0x7fb424005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb424003980 0x7fb424003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43d1cd640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 -- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 msgr2=0x7fb4240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb438070520 0x7fb4240047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 -- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb424003980 msgr2=0x7fb424003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb424003980 0x7fb424003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43e1cf640 1 -- 172.21.15.8:0/3003019843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424003530 con 0x7fb424004d10 2023-10-30T12:55:31.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.203+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb424003980 0x7fb424003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:31.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb424004d10 0x7fb424005190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb430009fc0 tx=0x7fb43001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7a5f3f5a1d65b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:31.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300a5c70 con 0x7fb424004d10 2023-10-30T12:55:31.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43000f040 con 0x7fb424004d10 2023-10-30T12:55:31.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43c9cc640 1 -- 172.21.15.8:0/3003019843 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300ad440 con 0x7fb424004d10 2023-10-30T12:55:31.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- 172.21.15.8:0/3003019843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb424004d10 msgr2=0x7fb424005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 --2- 172.21.15.8:0/3003019843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb424004d10 0x7fb424005190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb430009fc0 tx=0x7fb43001cc60 comp rx=0 tx=0).stop 2023-10-30T12:55:31.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43e1cf640 1 -- 172.21.15.8:0/3003019843 reap_dead start 2023-10-30T12:55:31.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- 172.21.15.8:0/3003019843 shutdown_connections 2023-10-30T12:55:31.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- 172.21.15.8:0/3003019843 >> 172.21.15.8:0/3003019843 conn(0x7fb4380cac70 msgr2=0x7fb4380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- 172.21.15.8:0/3003019843 shutdown_connections 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- 172.21.15.8:0/3003019843 wait complete. 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 Processor -- start 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.207+0000 7fb43fc32640 1 -- start start 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb424004d10 0x7fb4381f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 0x7fb438149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43814a210 con 0x7fb4380c5c90 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381ee6a0 con 0x7fb4380c5850 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381ee980 con 0x7fb424004d10 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.220 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32862/0 (socket says 172.21.15.8:32862) 2023-10-30T12:55:31.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 -- 172.21.15.8:0/2063365064 learned_addr learned my addr 172.21.15.8:0/2063365064 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:31.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb424004d10 0x7fb4381f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43d1cd640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 0x7fb438149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 -- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb424004d10 msgr2=0x7fb4381f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb424004d10 0x7fb4381f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 -- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 msgr2=0x7fb438149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 0x7fb438149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 -- 172.21.15.8:0/2063365064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4381eec60 con 0x7fb4380c5c90 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43d1cd640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 0x7fb438149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43e1cf640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb4300b0040 tx=0x7fb430009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:31.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300bc070 con 0x7fb4380c5c90 2023-10-30T12:55:31.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4300039a0 con 0x7fb4380c5c90 2023-10-30T12:55:31.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4381eef40 con 0x7fb4380c5c90 2023-10-30T12:55:31.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430003b30 con 0x7fb4380c5c90 2023-10-30T12:55:31.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb438149270 con 0x7fb4380c5c90 2023-10-30T12:55:31.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43804e170 con 0x7fb4380c5c90 2023-10-30T12:55:31.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.211+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb4300b4040 con 0x7fb4380c5c90 2023-10-30T12:55:31.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.215+0000 7fb42e7fc640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 0x7fb41c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:31.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.215+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fb43011e350 con 0x7fb4380c5c90 2023-10-30T12:55:31.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.215+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 0x7fb41c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:31.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.215+0000 7fb43d9ce640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 0x7fb41c068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb434006f90 tx=0x7fb434008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:31.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.219+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb4300eb510 con 0x7fb4380c5c90 2023-10-30T12:55:31.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb438070300 con 0x7fb4380c5c90 2023-10-30T12:55:31.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb42e7fc640 1 -- 172.21.15.8:0/2063365064 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5732 (secure 0 0 0) 0x7fb4300b2090 con 0x7fb4380c5c90 2023-10-30T12:55:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:31.412 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:28.938122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:31.412 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 55 2023-10-30T12:55:31.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 msgr2=0x7fb41c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 0x7fb41c068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb434006f90 tx=0x7fb434008040 comp rx=0 tx=0).stop 2023-10-30T12:55:31.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 msgr2=0x7fb438149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:31.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb4300b0040 tx=0x7fb430009750 comp rx=0 tx=0).stop 2023-10-30T12:55:31.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 shutdown_connections 2023-10-30T12:55:31.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fb41c0664a0 0x7fb41c068960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4380c5850 0x7fb438149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4380c5c90 0x7fb438149cd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 --2- 172.21.15.8:0/2063365064 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb424004d10 0x7fb4381f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:31.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 >> 172.21.15.8:0/2063365064 conn(0x7fb4380cac70 msgr2=0x7fb4380c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:31.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 shutdown_connections 2023-10-30T12:55:31.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:31.407+0000 7fb43fc32640 1 -- 172.21.15.8:0/2063365064 wait complete. 2023-10-30T12:55:31.432 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:33.434 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:33.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 Processor -- start 2023-10-30T12:55:33.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 -- start start 2023-10-30T12:55:33.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f481c151c20 0x7f481c0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c0cb8b0 0x7f481c06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481c0c5030 con 0x7f481c151840 2023-10-30T12:55:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c157830 con 0x7f481c151c20 2023-10-30T12:55:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f4821d9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c157b10 con 0x7f481c0cb8b0 2023-10-30T12:55:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f481affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f481c151c20 0x7f481c0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f481affd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f481c151c20 0x7f481c0c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32868/0 (socket says 172.21.15.8:32868) 2023-10-30T12:55:33.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f481affd640 1 -- 172.21.15.8:0/3601003701 learned_addr learned my addr 172.21.15.8:0/3601003701 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.583+0000 7f481bfff640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c0cb8b0 0x7f481c06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481affd640 1 -- 172.21.15.8:0/3601003701 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c0cb8b0 msgr2=0x7f481c06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481affd640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c0cb8b0 0x7f481c06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481affd640 1 -- 172.21.15.8:0/3601003701 >> v1:172.21.15.46:6789/0 conn(0x7f481c151840 legacy=0x7f481c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481affd640 1 -- 172.21.15.8:0/3601003701 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c157df0 con 0x7f481c151c20 2023-10-30T12:55:33.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481affd640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151c20 0x7f481c0c49a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4810009870 tx=0x7f481001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32c928219dda48e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:33.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481a7fc640 1 -- 172.21.15.8:0/3601003701 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100a8a30 con 0x7f481c151c20 2023-10-30T12:55:33.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481a7fc640 1 -- 172.21.15.8:0/3601003701 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100a8b90 con 0x7f481c151c20 2023-10-30T12:55:33.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f481a7fc640 1 -- 172.21.15.8:0/3601003701 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100b0440 con 0x7f481c151c20 2023-10-30T12:55:33.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 -- 172.21.15.8:0/3601003701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151c20 msgr2=0x7f481c0c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151c20 0x7f481c0c49a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4810009870 tx=0x7f481001ed30 comp rx=0 tx=0).stop 2023-10-30T12:55:33.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 -- 172.21.15.8:0/3601003701 shutdown_connections 2023-10-30T12:55:33.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c0cb8b0 0x7f481c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 --2- 172.21.15.8:0/3601003701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151c20 0x7f481c0c49a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.587+0000 7f4821d9d640 1 -- 172.21.15.8:0/3601003701 >> 172.21.15.8:0/3601003701 conn(0x7f481c0cac70 msgr2=0x7f481c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:33.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- 172.21.15.8:0/3601003701 shutdown_connections 2023-10-30T12:55:33.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- 172.21.15.8:0/3601003701 wait complete. 2023-10-30T12:55:33.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 Processor -- start 2023-10-30T12:55:33.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- start start 2023-10-30T12:55:33.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0cb8b0 0x7f481c145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c151c20 0x7f481c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c155d60 con 0x7f481c151840 2023-10-30T12:55:33.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c150800 con 0x7f481c0cb8b0 2023-10-30T12:55:33.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f4821d9d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c157200 con 0x7f481c151c20 2023-10-30T12:55:33.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32870/0 (socket says 172.21.15.8:32870) 2023-10-30T12:55:33.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 -- 172.21.15.8:0/1505941098 learned_addr learned my addr 172.21.15.8:0/1505941098 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:33.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481bfff640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c151c20 0x7f481c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481b7fe640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0cb8b0 0x7f481c145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 -- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c151c20 msgr2=0x7f481c152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c151c20 0x7f481c152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 -- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0cb8b0 msgr2=0x7f481c145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0cb8b0 0x7f481c145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.591+0000 7f481affd640 1 -- 172.21.15.8:0/1505941098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100094d0 con 0x7f481c151840 2023-10-30T12:55:33.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f481affd640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f48100b3040 tx=0x7f481001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:33.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100be070 con 0x7f481c151840 2023-10-30T12:55:33.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100ba400 con 0x7f481c151840 2023-10-30T12:55:33.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100ba7d0 con 0x7f481c151840 2023-10-30T12:55:33.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c155fd0 con 0x7f481c151840 2023-10-30T12:55:33.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c156590 con 0x7f481c151840 2023-10-30T12:55:33.605 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c156780 con 0x7f481c151840 2023-10-30T12:55:33.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48100c3410 con 0x7f481c151840 2023-10-30T12:55:33.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 0x7f4808068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:33.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f4810121c00 con 0x7f481c151840 2023-10-30T12:55:33.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.595+0000 7f481b7fe640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 0x7f4808068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:33.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.599+0000 7f481b7fe640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 0x7f4808068960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f70 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:33.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.599+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48100eeda0 con 0x7f481c151840 2023-10-30T12:55:33.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f481c066510 con 0x7f481c151840 2023-10-30T12:55:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4803fff640 1 -- 172.21.15.8:0/1505941098 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f48100b5090 con 0x7f481c151840 2023-10-30T12:55:33.799 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:33.800 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:28.938122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:33.800 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 55 2023-10-30T12:55:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 msgr2=0x7f4808068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 0x7f4808068960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f70 tx=0x7f480c009040 comp rx=0 tx=0).stop 2023-10-30T12:55:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 msgr2=0x7f481c152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f48100b3040 tx=0x7f481001bad0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 shutdown_connections 2023-10-30T12:55:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48080664a0 0x7f4808068960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f481c151c20 0x7f481c152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f481c151840 0x7f481c152030 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 --2- 172.21.15.8:0/1505941098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c0cb8b0 0x7f481c145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 >> 172.21.15.8:0/1505941098 conn(0x7f481c0cac70 msgr2=0x7f481c14f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 shutdown_connections 2023-10-30T12:55:33.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:33.795+0000 7f4821d9d640 1 -- 172.21.15.8:0/1505941098 wait complete. 2023-10-30T12:55:33.821 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:55:35.823 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:35.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.975+0000 7f0cb789c640 1 Processor -- start 2023-10-30T12:55:35.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb789c640 1 -- start start 2023-10-30T12:55:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb789c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0cb00c5ba0 0x7f0cb00c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:35.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb789c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb015cda0 con 0x7f0cb00c9b40 2023-10-30T12:55:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb789c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb006f490 con 0x7f0cb00c5f80 2023-10-30T12:55:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb789c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb015d990 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5638640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0cb00c5ba0 0x7f0cb00c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5e39640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0cb00c9b40 0x7f0cb01597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:43980/0 (socket says 172.21.15.8:43980) 2023-10-30T12:55:35.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5e39640 1 -- 172.21.15.8:0/1491886175 learned_addr learned my addr 172.21.15.8:0/1491886175 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5638640 1 -- 172.21.15.8:0/1491886175 >> v1:172.21.15.46:6789/0 conn(0x7f0cb00c5f80 legacy=0x7f0cb0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5638640 1 -- 172.21.15.8:0/1491886175 >> v1:172.21.15.8:6789/0 conn(0x7f0cb00c9b40 legacy=0x7f0cb01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:35.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0cb5638640 1 -- 172.21.15.8:0/1491886175 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb015cff0 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.979+0000 7f0ca7fff640 1 -- 172.21.15.8:0/1491886175 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149553865 0 0) 0x7f0cb015cda0 con 0x7f0cb00c9b40 2023-10-30T12:55:35.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb5638640 1 --2- 172.21.15.8:0/1491886175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c5ba0 0x7f0cb00c70f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097c0 tx=0x7f0cac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1402c474b354ff00 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:35.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0ca7fff640 1 -- 172.21.15.8:0/1491886175 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0ab600 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0ca7fff640 1 -- 172.21.15.8:0/1491886175 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0a7050 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0ca7fff640 1 -- 172.21.15.8:0/1491886175 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0ab980 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- 172.21.15.8:0/1491886175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c5ba0 msgr2=0x7f0cb00c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:35.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 --2- 172.21.15.8:0/1491886175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c5ba0 0x7f0cb00c70f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097c0 tx=0x7f0cac01c480 comp rx=0 tx=0).stop 2023-10-30T12:55:35.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- 172.21.15.8:0/1491886175 shutdown_connections 2023-10-30T12:55:35.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 --2- 172.21.15.8:0/1491886175 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c5ba0 0x7f0cb00c70f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:35.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- 172.21.15.8:0/1491886175 >> 172.21.15.8:0/1491886175 conn(0x7f0cb006afa0 msgr2=0x7f0cb006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:35.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- 172.21.15.8:0/1491886175 shutdown_connections 2023-10-30T12:55:35.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- 172.21.15.8:0/1491886175 wait complete. 2023-10-30T12:55:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 Processor -- start 2023-10-30T12:55:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.983+0000 7f0cb789c640 1 -- start start 2023-10-30T12:55:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0cb00c5ba0 0x7f0cb01f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00c5f80 0x7f0cb0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0156710 con 0x7f0cb00c9b40 2023-10-30T12:55:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb01f4850 con 0x7f0cb00c5f80 2023-10-30T12:55:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb01f3a10 con 0x7f0cb00c5ba0 2023-10-30T12:55:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32888/0 (socket says 172.21.15.8:32888) 2023-10-30T12:55:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 -- 172.21.15.8:0/774632174 learned_addr learned my addr 172.21.15.8:0/774632174 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5638640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0cb00c5ba0 0x7f0cb01f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb4e37640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00c5f80 0x7f0cb0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 -- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0cb00c5ba0 msgr2=0x7f0cb01f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:35.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0cb00c5ba0 0x7f0cb01f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:35.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 -- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00c5f80 msgr2=0x7f0cb0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:35.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00c5f80 0x7f0cb0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:35.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 -- 172.21.15.8:0/774632174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009420 con 0x7f0cb00c9b40 2023-10-30T12:55:35.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb5e39640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800c780 tx=0x7f0ca800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:35.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800ce30 con 0x7f0cb00c9b40 2023-10-30T12:55:35.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca800a640 con 0x7f0cb00c9b40 2023-10-30T12:55:35.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800a9f0 con 0x7f0cb00c9b40 2023-10-30T12:55:35.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb01f3cd0 con 0x7f0cb00c9b40 2023-10-30T12:55:35.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb01f4290 con 0x7f0cb00c9b40 2023-10-30T12:55:35.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb0154150 con 0x7f0cb00c9b40 2023-10-30T12:55:35.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.987+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0ca800ab50 con 0x7f0cb00c9b40 2023-10-30T12:55:35.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.991+0000 7f0ca5ffb640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 0x7f0c94068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:35.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.991+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f0ca8083410 con 0x7f0cb00c9b40 2023-10-30T12:55:35.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.991+0000 7f0cb5638640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 0x7f0c94068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:35.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.991+0000 7f0cb5638640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 0x7f0c94068990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0ad000 tx=0x7f0cac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:35.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:35.995+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0cb0154150 con 0x7f0cb00c9b40 2023-10-30T12:55:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.183+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cb0154150 con 0x7f0cb00c9b40 2023-10-30T12:55:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.183+0000 7f0ca5ffb640 1 -- 172.21.15.8:0/774632174 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f0cb0154150 con 0x7f0cb00c9b40 2023-10-30T12:55:36.188 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:36.189 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:34.258631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:36.189 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 56 2023-10-30T12:55:36.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 msgr2=0x7f0c94068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:36.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 0x7f0c94068990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0ad000 tx=0x7f0cac008040 comp rx=0 tx=0).stop 2023-10-30T12:55:36.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 msgr2=0x7f0cb0154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:36.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800c780 tx=0x7f0ca800cc40 comp rx=0 tx=0).stop 2023-10-30T12:55:36.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 shutdown_connections 2023-10-30T12:55:36.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0c940664d0 0x7f0c94068990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:36.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00c5f80 0x7f0cb0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:36.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0cb00c9b40 0x7f0cb0154430 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:36.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 --2- 172.21.15.8:0/774632174 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0cb00c5ba0 0x7f0cb01f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:36.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 >> 172.21.15.8:0/774632174 conn(0x7f0cb006afa0 msgr2=0x7f0cb0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:36.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 shutdown_connections 2023-10-30T12:55:36.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:36.187+0000 7f0cb789c640 1 -- 172.21.15.8:0/774632174 wait complete. 2023-10-30T12:55:36.209 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=4929 2023-10-30T12:55:36.210 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.c 2023-10-30T12:55:38.211 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-30T12:55:38.211 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-30T12:55:38.212 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-30T12:55:38.217 INFO:tasks.ceph.mds.c:Started 2023-10-30T12:55:38.217 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:38.314 INFO:tasks.ceph.mds.c.smithi008.stdout:starting mds.c at 2023-10-30T12:55:38.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.367+0000 7f6841005640 1 Processor -- start 2023-10-30T12:55:38.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.367+0000 7f6841005640 1 -- start start 2023-10-30T12:55:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6841005640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f683c0c4a10 0x7f683c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6841005640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 0x7f683c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6841005640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c159f20 con 0x7f683c153c80 2023-10-30T12:55:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6841005640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c15dec0 con 0x7f683c0c4a10 2023-10-30T12:55:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6841005640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c1599a0 con 0x7f683c0c42a0 2023-10-30T12:55:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f683c0c4a10 0x7f683c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f683c0c4a10 0x7f683c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32898/0 (socket says 172.21.15.8:32898) 2023-10-30T12:55:38.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 -- 172.21.15.8:0/2164484263 learned_addr learned my addr 172.21.15.8:0/2164484263 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:38.376 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f683b59e640 1 --2- 172.21.15.8:0/2164484263 >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 0x7f683c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 -- 172.21.15.8:0/2164484263 >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 msgr2=0x7f683c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:38.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 --2- 172.21.15.8:0/2164484263 >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 0x7f683c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 -- 172.21.15.8:0/2164484263 >> v1:172.21.15.46:6789/0 conn(0x7f683c153c80 legacy=0x7f683c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:55:38.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 -- 172.21.15.8:0/2164484263 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c15d620 con 0x7f683c0c4a10 2023-10-30T12:55:38.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f683b59e640 1 --2- 172.21.15.8:0/2164484263 >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 0x7f683c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:38.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f6833fff640 1 --2- 172.21.15.8:0/2164484263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c15cf90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f683c0b1d40 tx=0x7f682401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159fd16f085ef0a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:38.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f683a59c640 1 -- 172.21.15.8:0/2164484263 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a89b0 con 0x7f683c0c4a10 2023-10-30T12:55:38.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f683a59c640 1 -- 172.21.15.8:0/2164484263 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68240a8b10 con 0x7f683c0c4a10 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.371+0000 7f683a59c640 1 -- 172.21.15.8:0/2164484263 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a8ea0 con 0x7f683c0c4a10 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- 172.21.15.8:0/2164484263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 msgr2=0x7f683c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- 172.21.15.8:0/2164484263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c15cf90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f683c0b1d40 tx=0x7f682401ecb0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- 172.21.15.8:0/2164484263 shutdown_connections 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- 172.21.15.8:0/2164484263 >> v2:172.21.15.46:3300/0 conn(0x7f683c0c42a0 0x7f683c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- 172.21.15.8:0/2164484263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c15cf90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- 172.21.15.8:0/2164484263 >> 172.21.15.8:0/2164484263 conn(0x7f683c0715c0 msgr2=0x7f683c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:38.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- 172.21.15.8:0/2164484263 shutdown_connections 2023-10-30T12:55:38.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- 172.21.15.8:0/2164484263 wait complete. 2023-10-30T12:55:38.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 Processor -- start 2023-10-30T12:55:38.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- start start 2023-10-30T12:55:38.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c0c42a0 0x7f683c0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 0x7f683c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0b2390 con 0x7f683c0c4a10 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c14ef00 con 0x7f683c0c42a0 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6841005640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0b2130 con 0x7f683c153c80 2023-10-30T12:55:38.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6833fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6833fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32914/0 (socket says 172.21.15.8:32914) 2023-10-30T12:55:38.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f6833fff640 1 -- 172.21.15.8:0/899759998 learned_addr learned my addr 172.21.15.8:0/899759998 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:38.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f683ad9d640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c0c42a0 0x7f683c0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.375+0000 7f683b59e640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 0x7f683c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 -- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 msgr2=0x7f683c15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:38.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 0x7f683c15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 -- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c0c42a0 msgr2=0x7f683c0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:38.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c0c42a0 0x7f683c0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 -- 172.21.15.8:0/899759998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68240094d0 con 0x7f683c0c4a10 2023-10-30T12:55:38.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f683b59e640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 0x7f683c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:38.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6833fff640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f682401d360 tx=0x7f682401ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:38.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240b0e00 con 0x7f683c0c4a10 2023-10-30T12:55:38.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68240b9440 con 0x7f683c0c4a10 2023-10-30T12:55:38.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240b97f0 con 0x7f683c0c4a10 2023-10-30T12:55:38.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c14e0d0 con 0x7f683c0c4a10 2023-10-30T12:55:38.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c14e690 con 0x7f683c0c4a10 2023-10-30T12:55:38.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c04e170 con 0x7f683c0c4a10 2023-10-30T12:55:38.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.379+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68240b7070 con 0x7f683c0c4a10 2023-10-30T12:55:38.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.383+0000 7f68337fe640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 0x7f681c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:38.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.383+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f6824121d60 con 0x7f683c0c4a10 2023-10-30T12:55:38.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.383+0000 7f683ad9d640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 0x7f681c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:38.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.383+0000 7f683ad9d640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 0x7f681c068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:38.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.387+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68240eee70 con 0x7f683c0c4a10 2023-10-30T12:55:38.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f683c04e170 con 0x7f683c0c4a10 2023-10-30T12:55:38.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f68337fe640 1 -- 172.21.15.8:0/899759998 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f68240b5070 con 0x7f683c0c4a10 2023-10-30T12:55:38.583 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:38.583 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:34.258631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:38.584 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 56 2023-10-30T12:55:38.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 msgr2=0x7f681c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:38.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 0x7f681c068990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c009060 comp rx=0 tx=0).stop 2023-10-30T12:55:38.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 msgr2=0x7f683c14f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:38.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f682401d360 tx=0x7f682401ba50 comp rx=0 tx=0).stop 2023-10-30T12:55:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 shutdown_connections 2023-10-30T12:55:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f681c0664d0 0x7f681c068990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f683c153c80 0x7f683c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f683c0c4a10 0x7f683c14f390 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 --2- 172.21.15.8:0/899759998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c0c42a0 0x7f683c0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:38.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 >> 172.21.15.8:0/899759998 conn(0x7f683c0715c0 msgr2=0x7f683c15a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:38.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 shutdown_connections 2023-10-30T12:55:38.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:38.579+0000 7f6841005640 1 -- 172.21.15.8:0/899759998 wait complete. 2023-10-30T12:55:38.604 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-30T12:55:40.605 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:55:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 Processor -- start 2023-10-30T12:55:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 -- start start 2023-10-30T12:55:40.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7940151840 0x7f79400c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:40.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400c5030 con 0x7f7940151c20 2023-10-30T12:55:40.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7940157830 con 0x7f79400cb8b0 2023-10-30T12:55:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f7946254640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7940157b10 con 0x7f7940151840 2023-10-30T12:55:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f793effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7940151c20 0x7f7940072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:55264/0 (socket says 172.21.15.8:55264) 2023-10-30T12:55:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f793effd640 1 -- 172.21.15.8:0/955754856 learned_addr learned my addr 172.21.15.8:0/955754856 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:55:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f793f7fe640 1 --2- 172.21.15.8:0/955754856 >> v2:172.21.15.46:3300/0 conn(0x7f7940151840 0x7f79400c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365429227 0 0) 0x7f79400c5030 con 0x7f7940151c20 2023-10-30T12:55:40.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.755+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7928003530 con 0x7f7940151c20 2023-10-30T12:55:40.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793f7fe640 1 -- 172.21.15.8:0/955754856 >> v1:172.21.15.46:6789/0 conn(0x7f79400cb8b0 legacy=0x7f794006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840351947 0 0) 0x7f7928003530 con 0x7f7940151c20 2023-10-30T12:55:40.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793f7fe640 1 -- 172.21.15.8:0/955754856 >> v1:172.21.15.8:6789/0 conn(0x7f7940151c20 legacy=0x7f7940072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793f7fe640 1 -- 172.21.15.8:0/955754856 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940157df0 con 0x7f7940151840 2023-10-30T12:55:40.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793f7fe640 1 --2- 172.21.15.8:0/955754856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7940151840 0x7f79400c49a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f79300097c0 tx=0x7f793001eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7cc660f5985d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:40.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934001680 con 0x7f7940151840 2023-10-30T12:55:40.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79300a8610 con 0x7f7940151840 2023-10-30T12:55:40.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.759+0000 7f793e7fc640 1 -- 172.21.15.8:0/955754856 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300a89a0 con 0x7f7940151840 2023-10-30T12:55:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- 172.21.15.8:0/955754856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7940151840 msgr2=0x7f79400c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 --2- 172.21.15.8:0/955754856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7940151840 0x7f79400c49a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f79300097c0 tx=0x7f793001eb90 comp rx=0 tx=0).stop 2023-10-30T12:55:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- 172.21.15.8:0/955754856 shutdown_connections 2023-10-30T12:55:40.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 --2- 172.21.15.8:0/955754856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7940151840 0x7f79400c49a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- 172.21.15.8:0/955754856 >> 172.21.15.8:0/955754856 conn(0x7f79400cac70 msgr2=0x7f79400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:40.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- 172.21.15.8:0/955754856 shutdown_connections 2023-10-30T12:55:40.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- 172.21.15.8:0/955754856 wait complete. 2023-10-30T12:55:40.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 Processor -- start 2023-10-30T12:55:40.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- start start 2023-10-30T12:55:40.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79400cb8b0 0x7f794014afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:40.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:40.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 0x7f79400ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:40.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401ec750 con 0x7f7940151840 2023-10-30T12:55:40.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401efe40 con 0x7f79400cb8b0 2023-10-30T12:55:40.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f7946254640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79401f0120 con 0x7f7940151c20 2023-10-30T12:55:40.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:40.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39402/0 (socket says 172.21.15.8:39402) 2023-10-30T12:55:40.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 -- 172.21.15.8:0/2289945865 learned_addr learned my addr 172.21.15.8:0/2289945865 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:55:40.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793f7fe640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79400cb8b0 0x7f794014afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:40.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793ffff640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 0x7f79400ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:40.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 -- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 msgr2=0x7f79400ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 0x7f79400ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 -- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79400cb8b0 msgr2=0x7f794014afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79400cb8b0 0x7f794014afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793effd640 1 -- 172.21.15.8:0/2289945865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7930009420 con 0x7f7940151840 2023-10-30T12:55:40.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.763+0000 7f793ffff640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 0x7f79400ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:55:40.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f793effd640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f793400c780 tx=0x7f793400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:40.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934005ae0 con 0x7f7940151840 2023-10-30T12:55:40.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934005c40 con 0x7f7940151840 2023-10-30T12:55:40.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934007070 con 0x7f7940151840 2023-10-30T12:55:40.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79401f0400 con 0x7f7940151840 2023-10-30T12:55:40.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401f09c0 con 0x7f7940151840 2023-10-30T12:55:40.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794004e170 con 0x7f7940151840 2023-10-30T12:55:40.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f79340071d0 con 0x7f7940151840 2023-10-30T12:55:40.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 0x7f792c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:55:40.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f7934083de0 con 0x7f7940151840 2023-10-30T12:55:40.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f793f7fe640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 0x7f792c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:55:40.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.767+0000 7f793f7fe640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 0x7f792c068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f79300b2000 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:55:40.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.771+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7934051040 con 0x7f7940151840 2023-10-30T12:55:40.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7940146ed0 con 0x7f7940151840 2023-10-30T12:55:40.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7927fff640 1 -- 172.21.15.8:0/2289945865 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f7934054660 con 0x7f7940151840 2023-10-30T12:55:40.967 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:55:40.967 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:55:34.258631+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:55:40.968 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 57 2023-10-30T12:55:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 msgr2=0x7f792c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 0x7f792c068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f79300b2000 tx=0x7f7930008040 comp rx=0 tx=0).stop 2023-10-30T12:55:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 msgr2=0x7f79400c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:55:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f793400c780 tx=0x7f793400cc40 comp rx=0 tx=0).stop 2023-10-30T12:55:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 shutdown_connections 2023-10-30T12:55:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f792c0664a0 0x7f792c068960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7940151c20 0x7f79400ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7940151840 0x7f79400c9bd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 --2- 172.21.15.8:0/2289945865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79400cb8b0 0x7f794014afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:55:40.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 >> 172.21.15.8:0/2289945865 conn(0x7f79400cac70 msgr2=0x7f794014f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:55:40.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 shutdown_connections 2023-10-30T12:55:40.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:55:40.963+0000 7f7946254640 1 -- 172.21.15.8:0/2289945865 wait complete. 2023-10-30T12:55:40.988 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-30T12:55:40.988 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 68 secs before thrashing 2023-10-30T12:55:45.742 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:55:45.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:15.796 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:15.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:45.848 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:45.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:56:48.990 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:49.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.139+0000 7f5c030d6640 1 Processor -- start 2023-10-30T12:56:49.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.139+0000 7f5c030d6640 1 -- start start 2023-10-30T12:56:49.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.139+0000 7f5c030d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 0x7f5bfc153830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.139+0000 7f5c030d6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bfc06a4d0 0x7f5bfc153d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5c030d6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc0c1e60 con 0x7f5bfc0c0260 2023-10-30T12:56:49.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5c030d6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bfc15a3f0 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5c030d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0cb2b0 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bfc06a4d0 0x7f5bfc153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5c00e72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 0x7f5bfc153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bfc06a4d0 0x7f5bfc153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:36868/0 (socket says 172.21.15.8:36868) 2023-10-30T12:56:49.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 -- 172.21.15.8:0/3884124542 learned_addr learned my addr 172.21.15.8:0/3884124542 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:49.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5c00e72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 0x7f5bfc153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:46190/0 (socket says 172.21.15.8:46190) 2023-10-30T12:56:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 -- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 msgr2=0x7f5bfc153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 --2- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 0x7f5bfc153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 -- 172.21.15.8:0/3884124542 >> v1:172.21.15.46:6789/0 conn(0x7f5bfc0c0260 legacy=0x7f5bfc0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.143+0000 7f5bfbfff640 1 -- 172.21.15.8:0/3884124542 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc15a8c0 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5bfbfff640 1 --2- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc153d70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009870 tx=0x7f5be801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81cd341106f8109a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5bfb7fe640 1 -- 172.21.15.8:0/3884124542 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80aa670 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5bfb7fe640 1 -- 172.21.15.8:0/3884124542 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be80aa7d0 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5bfb7fe640 1 -- 172.21.15.8:0/3884124542 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80aab60 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 msgr2=0x7f5bfc153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 --2- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc153d70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009870 tx=0x7f5be801c510 comp rx=0 tx=0).stop 2023-10-30T12:56:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- 172.21.15.8:0/3884124542 shutdown_connections 2023-10-30T12:56:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 --2- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc153d70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 --2- 172.21.15.8:0/3884124542 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc06a0f0 0x7f5bfc153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- 172.21.15.8:0/3884124542 >> 172.21.15.8:0/3884124542 conn(0x7f5bfc06de90 msgr2=0x7f5bfc06e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:49.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- 172.21.15.8:0/3884124542 shutdown_connections 2023-10-30T12:56:49.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- 172.21.15.8:0/3884124542 wait complete. 2023-10-30T12:56:49.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 Processor -- start 2023-10-30T12:56:49.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.147+0000 7f5c030d6640 1 -- start start 2023-10-30T12:56:49.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc0c0260 0x7f5bfc14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1f2250 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc151a50 con 0x7f5bfc06a4d0 2023-10-30T12:56:49.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0c5010 con 0x7f5bfc0c0260 2023-10-30T12:56:49.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36984/0 (socket says 172.21.15.8:36984) 2023-10-30T12:56:49.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 -- 172.21.15.8:0/1582115422 learned_addr learned my addr 172.21.15.8:0/1582115422 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:49.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5bfbfff640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c01673640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc0c0260 0x7f5bfc14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 -- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc0c0260 msgr2=0x7f5bfc14cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc0c0260 0x7f5bfc14cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 -- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 msgr2=0x7f5bfc071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 -- 172.21.15.8:0/1582115422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80094d0 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c00e72640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e930 tx=0x7f5bf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0009690 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf000ca00 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc0c2560 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000cd70 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc0c52b0 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc04e170 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.151+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5bf0013470 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.155+0000 7f5bf97fa640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 0x7f5be0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.155+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f5bf0084270 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.155+0000 7f5bfbfff640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 0x7f5be0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.155+0000 7f5bfbfff640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 0x7f5be0068960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ae000 tx=0x7f5be80b5040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.159+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5bf0051610 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.347+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bfc14f170 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.347+0000 7f5bf97fa640 1 -- 172.21.15.8:0/1582115422 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f5bf0054c30 con 0x7f5bfc06a0f0 2023-10-30T12:56:49.353 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:49.354 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:49.354 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:49.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 msgr2=0x7f5be0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 0x7f5be0068960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ae000 tx=0x7f5be80b5040 comp rx=0 tx=0).stop 2023-10-30T12:56:49.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 msgr2=0x7f5bfc151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e930 tx=0x7f5bf000edf0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 shutdown_connections 2023-10-30T12:56:49.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f5be00664a0 0x7f5be0068960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bfc06a4d0 0x7f5bfc071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5bfc0c0260 0x7f5bfc14cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 --2- 172.21.15.8:0/1582115422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bfc06a0f0 0x7f5bfc151510 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 >> 172.21.15.8:0/1582115422 conn(0x7f5bfc06de90 msgr2=0x7f5bfc14dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:49.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 shutdown_connections 2023-10-30T12:56:49.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.351+0000 7f5c030d6640 1 -- 172.21.15.8:0/1582115422 wait complete. 2023-10-30T12:56:49.374 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2023-10-30T12:56:49.374 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:56:49.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:56:49.393 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:56:49.394 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.575+0000 7fe5f1009640 1 Processor -- start 2023-10-30T12:56:49.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.575+0000 7fe5f1009640 1 -- start start 2023-10-30T12:56:49.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5f1009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5ec06f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5f1009640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec0ca180 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5f1009640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec0ca460 con 0x7fe5ec0c8060 2023-10-30T12:56:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5f1009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec150d00 con 0x7fe5ec0ca9b0 2023-10-30T12:56:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5e3fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe5ec0cb8b0 0x7fe5ec15ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35820/0 (socket says 172.21.15.8:35820) 2023-10-30T12:56:49.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5e3fff640 1 -- 172.21.15.8:0/1252906124 learned_addr learned my addr 172.21.15.8:0/1252906124 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:56:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5eb59e640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5ec06f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428549788 0 0) 0x7fe5ec0ca180 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d8005740 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1157887168 0 0) 0x7fe5d8005740 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 msgr2=0x7fe5ec06f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:49.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5ec06f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 >> v1:172.21.15.46:6789/0 conn(0x7fe5ec0c8060 legacy=0x7fe5ec0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec06e7b0 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fe5d40016a0 con 0x7fe5ec0cb8b0 2023-10-30T12:56:49.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 >> v1:172.21.15.8:6789/0 conn(0x7fe5ec0cb8b0 legacy=0x7fe5ec15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5d8005bc0 0x7fe5d8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5d80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8006f20 0x7fe5d80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec150d00 con 0x7fe5d8005bc0 2023-10-30T12:56:49.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d800dce0 con 0x7fe5d8006f20 2023-10-30T12:56:49.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d800dea0 con 0x7fe5ec0ca9b0 2023-10-30T12:56:49.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5d8005bc0 0x7fe5d8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5eb59e640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8006f20 0x7fe5d80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5e3fff640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5d80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 -- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 msgr2=0x7fe5d80069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:49.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec0ca9b0 0x7fe5d80069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 -- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8006f20 msgr2=0x7fe5d80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:49.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8006f20 0x7fe5d80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5ead9d640 1 -- 172.21.15.8:0/1252906124 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8005740 con 0x7fe5d8005bc0 2023-10-30T12:56:49.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.579+0000 7fe5eb59e640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8006f20 0x7fe5d80073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:49.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5d8005bc0 0x7fe5d8005fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc001750 tx=0x7fe5dc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8811bfbe6dd747 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0052d0 con 0x7fe5d8005bc0 2023-10-30T12:56:49.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc0a7050 con 0x7fe5d8005bc0 2023-10-30T12:56:49.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5ea59c640 1 -- 172.21.15.8:0/1252906124 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc003600 con 0x7fe5d8005bc0 2023-10-30T12:56:49.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- 172.21.15.8:0/1252906124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5d8005bc0 msgr2=0x7fe5d8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 --2- 172.21.15.8:0/1252906124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5d8005bc0 0x7fe5d8005fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc001750 tx=0x7fe5dc01c3c0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5eb59e640 1 -- 172.21.15.8:0/1252906124 reap_dead start 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- 172.21.15.8:0/1252906124 shutdown_connections 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- 172.21.15.8:0/1252906124 >> 172.21.15.8:0/1252906124 conn(0x7fe5ec06afa0 msgr2=0x7fe5ec06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- 172.21.15.8:0/1252906124 shutdown_connections 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- 172.21.15.8:0/1252906124 wait complete. 2023-10-30T12:56:49.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 Processor -- start 2023-10-30T12:56:49.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.583+0000 7fe5f1009640 1 -- start start 2023-10-30T12:56:49.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8005bc0 0x7fe5ec156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 0x7fe5ec153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec156a70 con 0x7fe5ec1570b0 2023-10-30T12:56:49.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec154f90 con 0x7fe5d8005bc0 2023-10-30T12:56:49.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1ef950 con 0x7fe5ec155420 2023-10-30T12:56:49.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36996/0 (socket says 172.21.15.8:36996) 2023-10-30T12:56:49.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 -- 172.21.15.8:0/2134469057 learned_addr learned my addr 172.21.15.8:0/2134469057 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:49.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8005bc0 0x7fe5ec156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5eb59e640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 0x7fe5ec153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 -- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 msgr2=0x7fe5ec153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 0x7fe5ec153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 -- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8005bc0 msgr2=0x7fe5ec156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8005bc0 0x7fe5ec156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 -- 172.21.15.8:0/2134469057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec1efbf0 con 0x7fe5ec1570b0 2023-10-30T12:56:49.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5eb59e640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 0x7fe5ec153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:49.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e3fff640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4011870 tx=0x7fe5d4011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d401a030 con 0x7fe5ec1570b0 2023-10-30T12:56:49.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400fb00 con 0x7fe5ec1570b0 2023-10-30T12:56:49.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4015410 con 0x7fe5ec1570b0 2023-10-30T12:56:49.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec154660 con 0x7fe5ec1570b0 2023-10-30T12:56:49.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec154c20 con 0x7fe5ec1570b0 2023-10-30T12:56:49.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec04e170 con 0x7fe5ec1570b0 2023-10-30T12:56:49.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.587+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe5d401d070 con 0x7fe5ec1570b0 2023-10-30T12:56:49.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.591+0000 7fe5e37fe640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 0x7fe5cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:49.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.591+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fe5d401b070 con 0x7fe5ec1570b0 2023-10-30T12:56:49.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.591+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 0x7fe5cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:49.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.591+0000 7fe5ead9d640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 0x7fe5cc068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80079c0 tx=0x7fe5dc0011a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:49.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.595+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe5d4054250 con 0x7fe5ec1570b0 2023-10-30T12:56:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.783+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5ec06ea60 con 0x7fe5ec1570b0 2023-10-30T12:56:49.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.783+0000 7fe5e37fe640 1 -- 172.21.15.8:0/2134469057 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7fe5d4057870 con 0x7fe5ec1570b0 2023-10-30T12:56:49.789 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:49.789 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:49.790 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 msgr2=0x7fe5cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 0x7fe5cc068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80079c0 tx=0x7fe5dc0011a0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 msgr2=0x7fe5ec14e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4011870 tx=0x7fe5d4011d30 comp rx=0 tx=0).stop 2023-10-30T12:56:49.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 shutdown_connections 2023-10-30T12:56:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe5cc0664a0 0x7fe5cc068960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe5ec155420 0x7fe5ec153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe5ec1570b0 0x7fe5ec14e3d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 --2- 172.21.15.8:0/2134469057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d8005bc0 0x7fe5ec156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:49.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 >> 172.21.15.8:0/2134469057 conn(0x7fe5ec06afa0 msgr2=0x7fe5ec0c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 shutdown_connections 2023-10-30T12:56:49.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:49.787+0000 7fe5f1009640 1 -- 172.21.15.8:0/2134469057 wait complete. 2023-10-30T12:56:49.810 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:56:51.811 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:51.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 Processor -- start 2023-10-30T12:56:51.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 -- start start 2023-10-30T12:56:51.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5314151840 0x7f53140c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f5314072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53140c5030 con 0x7f53140cb8b0 2023-10-30T12:56:51.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5314157830 con 0x7f5314151840 2023-10-30T12:56:51.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f531b1ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5314157b10 con 0x7f5314151c20 2023-10-30T12:56:51.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5314151840 0x7f53140c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5319748640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f53140cb8b0 0x7f531406b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35832/0 (socket says 172.21.15.8:35832) 2023-10-30T12:56:51.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5314151840 0x7f53140c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37002/0 (socket says 172.21.15.8:37002) 2023-10-30T12:56:51.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5319748640 1 -- 172.21.15.8:0/4063251706 learned_addr learned my addr 172.21.15.8:0/4063251706 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:56:51.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5313fff640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f5314072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 -- 172.21.15.8:0/4063251706 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 msgr2=0x7f5314072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:51.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f5314072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:51.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 -- 172.21.15.8:0/4063251706 >> v1:172.21.15.8:6789/0 conn(0x7f53140cb8b0 legacy=0x7f531406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:51.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f5318f47640 1 -- 172.21.15.8:0/4063251706 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314157df0 con 0x7f5314151840 2023-10-30T12:56:51.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.963+0000 7f53137fe640 1 -- 172.21.15.8:0/4063251706 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652038662 0 0) 0x7f53140c5030 con 0x7f53140cb8b0 2023-10-30T12:56:51.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f5318f47640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53140c49a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f53080097c0 tx=0x7f530801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929503411fbf95b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:51.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f53137fe640 1 -- 172.21.15.8:0/4063251706 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080a8970 con 0x7f5314151840 2023-10-30T12:56:51.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f53137fe640 1 -- 172.21.15.8:0/4063251706 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53080a8ad0 con 0x7f5314151840 2023-10-30T12:56:51.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f53137fe640 1 -- 172.21.15.8:0/4063251706 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53080a8e60 con 0x7f5314151840 2023-10-30T12:56:51.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 -- 172.21.15.8:0/4063251706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 msgr2=0x7f53140c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:51.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53140c49a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f53080097c0 tx=0x7f530801ec70 comp rx=0 tx=0).stop 2023-10-30T12:56:51.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 -- 172.21.15.8:0/4063251706 shutdown_connections 2023-10-30T12:56:51.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f5314072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:51.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 --2- 172.21.15.8:0/4063251706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53140c49a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:51.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 -- 172.21.15.8:0/4063251706 >> 172.21.15.8:0/4063251706 conn(0x7f53140cac70 msgr2=0x7f53140cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:51.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 -- 172.21.15.8:0/4063251706 shutdown_connections 2023-10-30T12:56:51.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.967+0000 7f531b1ab640 1 -- 172.21.15.8:0/4063251706 wait complete. 2023-10-30T12:56:51.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 Processor -- start 2023-10-30T12:56:51.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- start start 2023-10-30T12:56:51.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 0x7f53141f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f53140c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53141efa90 con 0x7f5314151840 2023-10-30T12:56:51.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140c9d90 con 0x7f53140cb8b0 2023-10-30T12:56:51.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53141f2150 con 0x7f5314151c20 2023-10-30T12:56:51.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37014/0 (socket says 172.21.15.8:37014) 2023-10-30T12:56:51.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 -- 172.21.15.8:0/916558579 learned_addr learned my addr 172.21.15.8:0/916558579 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:51.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5319748640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f53140c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5318f47640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 0x7f53141f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 -- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 msgr2=0x7f53140c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f53140c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 -- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 msgr2=0x7f53141f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 0x7f53141f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 -- 172.21.15.8:0/916558579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308009420 con 0x7f5314151840 2023-10-30T12:56:51.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5319748640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f53140c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:51.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5318f47640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 0x7f53141f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:51.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f5313fff640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f530000d7d0 tx=0x7f530000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:51.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000b6c0 con 0x7f5314151840 2023-10-30T12:56:51.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000b820 con 0x7f5314151840 2023-10-30T12:56:51.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000bbb0 con 0x7f5314151840 2023-10-30T12:56:51.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5314073c50 con 0x7f5314151840 2023-10-30T12:56:51.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53140ca960 con 0x7f5314151840 2023-10-30T12:56:51.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.971+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531404e170 con 0x7f5314151840 2023-10-30T12:56:51.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.975+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f530000bd10 con 0x7f5314151840 2023-10-30T12:56:51.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.975+0000 7f53117fa640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 0x7f52f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:51.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.975+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f53000845e0 con 0x7f5314151840 2023-10-30T12:56:51.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.975+0000 7f5318f47640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 0x7f52f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:51.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.975+0000 7f5318f47640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 0x7f52f8068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f53080b2000 tx=0x7f5308008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:51.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:51.979+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5300051780 con 0x7f5314151840 2023-10-30T12:56:52.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.171+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5314155060 con 0x7f5314151840 2023-10-30T12:56:52.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.171+0000 7f53117fa640 1 -- 172.21.15.8:0/916558579 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f5300054da0 con 0x7f5314151840 2023-10-30T12:56:52.177 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:52.177 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:52.177 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:52.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 msgr2=0x7f52f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:52.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 0x7f52f8068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f53080b2000 tx=0x7f5308008040 comp rx=0 tx=0).stop 2023-10-30T12:56:52.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 msgr2=0x7f53141dda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:52.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f530000d7d0 tx=0x7f530000dc90 comp rx=0 tx=0).stop 2023-10-30T12:56:52.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 shutdown_connections 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f52f80664a0 0x7f52f8068960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f5314151c20 0x7f53140c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5314151840 0x7f53141dda60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 --2- 172.21.15.8:0/916558579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53140cb8b0 0x7f53141f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 >> 172.21.15.8:0/916558579 conn(0x7f53140cac70 msgr2=0x7f53140cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:52.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 shutdown_connections 2023-10-30T12:56:52.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:52.175+0000 7f531b1ab640 1 -- 172.21.15.8:0/916558579 wait complete. 2023-10-30T12:56:52.198 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:56:54.199 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:54.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 Processor -- start 2023-10-30T12:56:54.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 -- start start 2023-10-30T12:56:54.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0b8153c80 0x7fa0b815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa0b80c4a10 0x7fa0b80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b8159f20 con 0x7fa0b815a3b0 2023-10-30T12:56:54.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b815dec0 con 0x7fa0b80c4a10 2023-10-30T12:56:54.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bf101640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b81599a0 con 0x7fa0b8153c80 2023-10-30T12:56:54.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa0b80c4a10 0x7fa0b80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa0b80c4a10 0x7fa0b80c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37020/0 (socket says 172.21.15.8:37020) 2023-10-30T12:56:54.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 -- 172.21.15.8:0/2618201672 learned_addr learned my addr 172.21.15.8:0/2618201672 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:54.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0bce9d640 1 --2- 172.21.15.8:0/2618201672 >> v2:172.21.15.46:3300/0 conn(0x7fa0b8153c80 0x7fa0b815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 -- 172.21.15.8:0/2618201672 >> v2:172.21.15.46:3300/0 conn(0x7fa0b8153c80 msgr2=0x7fa0b815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:54.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 --2- 172.21.15.8:0/2618201672 >> v2:172.21.15.46:3300/0 conn(0x7fa0b8153c80 0x7fa0b815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 -- 172.21.15.8:0/2618201672 >> v1:172.21.15.46:6789/0 conn(0x7fa0b815a3b0 legacy=0x7fa0b80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:54.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.351+0000 7fa0b7fff640 1 -- 172.21.15.8:0/2618201672 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80c47e0 con 0x7fa0b80c4a10 2023-10-30T12:56:54.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0b7fff640 1 --2- 172.21.15.8:0/2618201672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b80c4a10 0x7fa0b80c42a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80d00f0 tx=0x7fa0a401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9176febdf35d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:54.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0b77fe640 1 -- 172.21.15.8:0/2618201672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40a89b0 con 0x7fa0b80c4a10 2023-10-30T12:56:54.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0b77fe640 1 -- 172.21.15.8:0/2618201672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a40a8b10 con 0x7fa0b80c4a10 2023-10-30T12:56:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0b77fe640 1 -- 172.21.15.8:0/2618201672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40a8ea0 con 0x7fa0b80c4a10 2023-10-30T12:56:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 -- 172.21.15.8:0/2618201672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b80c4a10 msgr2=0x7fa0b80c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 --2- 172.21.15.8:0/2618201672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b80c4a10 0x7fa0b80c42a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80d00f0 tx=0x7fa0a401ecb0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 -- 172.21.15.8:0/2618201672 shutdown_connections 2023-10-30T12:56:54.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 --2- 172.21.15.8:0/2618201672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b80c4a10 0x7fa0b80c42a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 --2- 172.21.15.8:0/2618201672 >> v2:172.21.15.46:3300/0 conn(0x7fa0b8153c80 0x7fa0b815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 -- 172.21.15.8:0/2618201672 >> 172.21.15.8:0/2618201672 conn(0x7fa0b80715c0 msgr2=0x7fa0b80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 -- 172.21.15.8:0/2618201672 shutdown_connections 2023-10-30T12:56:54.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.355+0000 7fa0bf101640 1 -- 172.21.15.8:0/2618201672 wait complete. 2023-10-30T12:56:54.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 Processor -- start 2023-10-30T12:56:54.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- start start 2023-10-30T12:56:54.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0b80c4a10 0x7fa0b81ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 0x7fa0b80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81f2e20 con 0x7fa0b8153c80 2023-10-30T12:56:54.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81ef930 con 0x7fa0b815a3b0 2023-10-30T12:56:54.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81f2990 con 0x7fa0b80c4a10 2023-10-30T12:56:54.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37026/0 (socket says 172.21.15.8:37026) 2023-10-30T12:56:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 -- 172.21.15.8:0/3026884166 learned_addr learned my addr 172.21.15.8:0/3026884166 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bce9d640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0b80c4a10 0x7fa0b81ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bd69e640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 0x7fa0b80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 -- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0b80c4a10 msgr2=0x7fa0b81ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0b80c4a10 0x7fa0b81ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 -- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 msgr2=0x7fa0b80b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 0x7fa0b80b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 -- 172.21.15.8:0/3026884166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40094d0 con 0x7fa0b8153c80 2023-10-30T12:56:54.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bd69e640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 0x7fa0b80b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:56:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b7fff640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40b3040 tx=0x7fa0a401ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40b0e00 con 0x7fa0b8153c80 2023-10-30T12:56:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a40bb470 con 0x7fa0b8153c80 2023-10-30T12:56:54.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40bb820 con 0x7fa0b8153c80 2023-10-30T12:56:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b8150ae0 con 0x7fa0b8153c80 2023-10-30T12:56:54.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.359+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81f2060 con 0x7fa0b8153c80 2023-10-30T12:56:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804e170 con 0x7fa0b8153c80 2023-10-30T12:56:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa0a40b7070 con 0x7fa0b8153c80 2023-10-30T12:56:54.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0b57fa640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 0x7fa09c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fa0a4122480 con 0x7fa0b8153c80 2023-10-30T12:56:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0bce9d640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 0x7fa09c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.363+0000 7fa0bce9d640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 0x7fa09c068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b9d0 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:54.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.367+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa0a40ef620 con 0x7fa0b8153c80 2023-10-30T12:56:54.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.555+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0b804e170 con 0x7fa0b8153c80 2023-10-30T12:56:54.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0b57fa640 1 -- 172.21.15.8:0/3026884166 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7fa0a40f9030 con 0x7fa0b8153c80 2023-10-30T12:56:54.562 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:54.563 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:54.563 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:54.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 msgr2=0x7fa09c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:54.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 0x7fa09c068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b9d0 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2023-10-30T12:56:54.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 msgr2=0x7fa0b81505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:54.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40b3040 tx=0x7fa0a401ba50 comp rx=0 tx=0).stop 2023-10-30T12:56:54.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 shutdown_connections 2023-10-30T12:56:54.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa09c0664a0 0x7fa09c068960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b815a3b0 0x7fa0b80b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0b8153c80 0x7fa0b81505a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 --2- 172.21.15.8:0/3026884166 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa0b80c4a10 0x7fa0b81ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:54.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 >> 172.21.15.8:0/3026884166 conn(0x7fa0b80715c0 msgr2=0x7fa0b8152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:54.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 shutdown_connections 2023-10-30T12:56:54.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:54.559+0000 7fa0bf101640 1 -- 172.21.15.8:0/3026884166 wait complete. 2023-10-30T12:56:54.584 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:56:56.585 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:56.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 Processor -- start 2023-10-30T12:56:56.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 -- start start 2023-10-30T12:56:56.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f48fc0724c0 0x7f48fc158a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:56.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc158110 con 0x7f48fc0720e0 2023-10-30T12:56:56.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc1583f0 con 0x7f48fc157c90 2023-10-30T12:56:56.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f4901002640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc0c1f80 con 0x7f48fc0724c0 2023-10-30T12:56:56.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fb59e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f48fc0724c0 0x7f48fc158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:56.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fad9d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f48fc0720e0 0x7f48fc157580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:35840/0 (socket says 172.21.15.8:35840) 2023-10-30T12:56:56.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fad9d640 1 -- 172.21.15.8:0/1695055006 learned_addr learned my addr 172.21.15.8:0/1695055006 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:56:56.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fb59e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f48fc0724c0 0x7f48fc158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37030/0 (socket says 172.21.15.8:37030) 2023-10-30T12:56:56.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fb59e640 1 -- 172.21.15.8:0/1695055006 >> v1:172.21.15.46:6789/0 conn(0x7f48fc157c90 legacy=0x7f48fc150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fb59e640 1 -- 172.21.15.8:0/1695055006 >> v1:172.21.15.8:6789/0 conn(0x7f48fc0720e0 legacy=0x7f48fc157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.739+0000 7f48fb59e640 1 -- 172.21.15.8:0/1695055006 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0c2260 con 0x7f48fc0724c0 2023-10-30T12:56:56.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f48fb59e640 1 --2- 172.21.15.8:0/1695055006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0724c0 0x7f48fc158a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d90 tx=0x7f48f401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95da9d92307b0f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:56.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f48fa59c640 1 -- 172.21.15.8:0/1695055006 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40af600 con 0x7f48fc0724c0 2023-10-30T12:56:56.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f48fa59c640 1 -- 172.21.15.8:0/1695055006 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40ab050 con 0x7f48fc0724c0 2023-10-30T12:56:56.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f48fa59c640 1 -- 172.21.15.8:0/1695055006 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40af980 con 0x7f48fc0724c0 2023-10-30T12:56:56.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- 172.21.15.8:0/1695055006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0724c0 msgr2=0x7f48fc158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 --2- 172.21.15.8:0/1695055006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0724c0 0x7f48fc158a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009d90 tx=0x7f48f401f130 comp rx=0 tx=0).stop 2023-10-30T12:56:56.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- 172.21.15.8:0/1695055006 shutdown_connections 2023-10-30T12:56:56.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 --2- 172.21.15.8:0/1695055006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0724c0 0x7f48fc158a70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- 172.21.15.8:0/1695055006 >> 172.21.15.8:0/1695055006 conn(0x7f48fc06d810 msgr2=0x7f48fc06dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:56.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- 172.21.15.8:0/1695055006 shutdown_connections 2023-10-30T12:56:56.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- 172.21.15.8:0/1695055006 wait complete. 2023-10-30T12:56:56.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 Processor -- start 2023-10-30T12:56:56.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.743+0000 7f4901002640 1 -- start start 2023-10-30T12:56:56.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:56.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc0724c0 0x7f48fc14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:56.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 0x7f48fc14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:56.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc14ea00 con 0x7f48fc0720e0 2023-10-30T12:56:56.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc1f2b20 con 0x7f48fc0724c0 2023-10-30T12:56:56.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc06fc70 con 0x7f48fc157c90 2023-10-30T12:56:56.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:56.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37040/0 (socket says 172.21.15.8:37040) 2023-10-30T12:56:56.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 -- 172.21.15.8:0/791260700 learned_addr learned my addr 172.21.15.8:0/791260700 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:56.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48f3fff640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc0724c0 0x7f48fc14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:56.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fb59e640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 0x7f48fc14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:56.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 -- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 msgr2=0x7f48fc14ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 0x7f48fc14ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 -- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc0724c0 msgr2=0x7f48fc14be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc0724c0 0x7f48fc14be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 -- 172.21.15.8:0/791260700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4009a70 con 0x7f48fc0720e0 2023-10-30T12:56:56.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fb59e640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 0x7f48fc14ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:56:56.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48fad9d640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c500 tx=0x7f48ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:56.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec005cf0 con 0x7f48fc0720e0 2023-10-30T12:56:56.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec005e50 con 0x7f48fc0720e0 2023-10-30T12:56:56.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc06ff50 con 0x7f48fc0720e0 2023-10-30T12:56:56.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0072f0 con 0x7f48fc0720e0 2023-10-30T12:56:56.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc070510 con 0x7f48fc0720e0 2023-10-30T12:56:56.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.747+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc04e170 con 0x7f48fc0720e0 2023-10-30T12:56:56.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.751+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48ec007450 con 0x7f48fc0720e0 2023-10-30T12:56:56.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.751+0000 7f48f37fe640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 0x7f48dc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:56.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.751+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f48ec083520 con 0x7f48fc0720e0 2023-10-30T12:56:56.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.751+0000 7f48f3fff640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 0x7f48dc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:56.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.751+0000 7f48f3fff640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 0x7f48dc068990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48fc143120 tx=0x7f48e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:56.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.755+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48ec050800 con 0x7f48fc0720e0 2023-10-30T12:56:56.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48fc0c27f0 con 0x7f48fc0720e0 2023-10-30T12:56:56.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f48f37fe640 1 -- 172.21.15.8:0/791260700 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f48ec053e20 con 0x7f48fc0720e0 2023-10-30T12:56:56.964 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:56.964 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:56.965 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 msgr2=0x7f48dc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 0x7f48dc068990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48fc143120 tx=0x7f48e4009040 comp rx=0 tx=0).stop 2023-10-30T12:56:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 msgr2=0x7f48fc1e3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.959+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c500 tx=0x7f48ec00c9c0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 shutdown_connections 2023-10-30T12:56:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f48dc0664d0 0x7f48dc068990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc0724c0 0x7f48fc14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f48fc157c90 0x7f48fc14ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 --2- 172.21.15.8:0/791260700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48fc0720e0 0x7f48fc1e3e50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:56.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 >> 172.21.15.8:0/791260700 conn(0x7f48fc06d810 msgr2=0x7f48fc159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:56.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 shutdown_connections 2023-10-30T12:56:56.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:56.963+0000 7f4901002640 1 -- 172.21.15.8:0/791260700 wait complete. 2023-10-30T12:56:56.985 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:56:58.986 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:56:59.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.134+0000 7f59275aa640 1 Processor -- start 2023-10-30T12:56:59.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.134+0000 7f59275aa640 1 -- start start 2023-10-30T12:56:59.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f59275aa640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f59201516c0 0x7f59200c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f59275aa640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c2500 0x7f59200c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f59275aa640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5920151aa0 con 0x7f59200c82d0 2023-10-30T12:56:59.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f59275aa640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200c74a0 con 0x7f59201516c0 2023-10-30T12:56:59.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f59275aa640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c2e70 con 0x7f59200c2500 2023-10-30T12:56:59.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f59201516c0 0x7f59200c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f59201516c0 0x7f59200c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39696/0 (socket says 172.21.15.8:39696) 2023-10-30T12:56:59.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 -- 172.21.15.8:0/2503203458 learned_addr learned my addr 172.21.15.8:0/2503203458 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:59.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5924b45640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f59200c82d0 0x7f59200ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39368/0 (socket says 172.21.15.8:39368) 2023-10-30T12:56:59.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925b47640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c2500 0x7f59200c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 -- 172.21.15.8:0/2503203458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c2500 msgr2=0x7f59200c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:56:59.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c2500 0x7f59200c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 -- 172.21.15.8:0/2503203458 >> v1:172.21.15.8:6789/0 conn(0x7f59200c82d0 legacy=0x7f59200ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5925346640 1 -- 172.21.15.8:0/2503203458 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920158af0 con 0x7f59201516c0 2023-10-30T12:56:59.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.138+0000 7f5917fff640 1 -- 172.21.15.8:0/2503203458 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002537482 0 0) 0x7f5920151aa0 con 0x7f59200c82d0 2023-10-30T12:56:59.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f5925346640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59200c7d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f591c009870 tx=0x7f591c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66617bbc7a80eb40 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f5917fff640 1 -- 172.21.15.8:0/2503203458 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0ab600 con 0x7f59201516c0 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f5917fff640 1 -- 172.21.15.8:0/2503203458 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c0a7050 con 0x7f59201516c0 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f5917fff640 1 -- 172.21.15.8:0/2503203458 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0ab980 con 0x7f59201516c0 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- 172.21.15.8:0/2503203458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 msgr2=0x7f59200c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59200c7d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f591c009870 tx=0x7f591c01c540 comp rx=0 tx=0).stop 2023-10-30T12:56:59.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- 172.21.15.8:0/2503203458 shutdown_connections 2023-10-30T12:56:59.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c2500 0x7f59200c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 --2- 172.21.15.8:0/2503203458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59200c7d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- 172.21.15.8:0/2503203458 >> 172.21.15.8:0/2503203458 conn(0x7f5920071fb0 msgr2=0x7f59200723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:59.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- 172.21.15.8:0/2503203458 shutdown_connections 2023-10-30T12:56:59.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- 172.21.15.8:0/2503203458 wait complete. 2023-10-30T12:56:59.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 Processor -- start 2023-10-30T12:56:59.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.142+0000 7f59275aa640 1 -- start start 2023-10-30T12:56:59.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59200c2500 0x7f592006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c82d0 0x7f59201f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920145b40 con 0x7f59201516c0 2023-10-30T12:56:59.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592015a060 con 0x7f59200c2500 2023-10-30T12:56:59.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5920159ae0 con 0x7f59200c82d0 2023-10-30T12:56:59.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39708/0 (socket says 172.21.15.8:39708) 2023-10-30T12:56:59.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 -- 172.21.15.8:0/2341085271 learned_addr learned my addr 172.21.15.8:0/2341085271 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:56:59.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5924b45640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c82d0 0x7f59201f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925346640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59200c2500 0x7f592006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 -- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c82d0 msgr2=0x7f59201f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c82d0 0x7f59201f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 -- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59200c2500 msgr2=0x7f592006bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59200c2500 0x7f592006bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 -- 172.21.15.8:0/2341085271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0094d0 con 0x7f59201516c0 2023-10-30T12:56:59.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5925b47640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5918007e60 tx=0x7f591800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:59.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918014420 con 0x7f59201516c0 2023-10-30T12:56:59.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59180145a0 con 0x7f59201516c0 2023-10-30T12:56:59.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918014950 con 0x7f59201516c0 2023-10-30T12:56:59.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592015a320 con 0x7f59201516c0 2023-10-30T12:56:59.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5920159d80 con 0x7f59201516c0 2023-10-30T12:56:59.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592004e170 con 0x7f59201516c0 2023-10-30T12:56:59.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.146+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5918014ab0 con 0x7f59201516c0 2023-10-30T12:56:59.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.150+0000 7f5915ffb640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 0x7f5904068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:56:59.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.150+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f5918084a00 con 0x7f59201516c0 2023-10-30T12:56:59.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.150+0000 7f5925346640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 0x7f5904068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:56:59.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.150+0000 7f5925346640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 0x7f5904068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59200caae0 tx=0x7f591c0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:56:59.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.154+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5918051c30 con 0x7f59201516c0 2023-10-30T12:56:59.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5920154f20 con 0x7f59201516c0 2023-10-30T12:56:59.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f5915ffb640 1 -- 172.21.15.8:0/2341085271 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f5918055250 con 0x7f59201516c0 2023-10-30T12:56:59.352 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:56:59.353 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:56:59.354 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:56:59.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 msgr2=0x7f5904068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 0x7f5904068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59200caae0 tx=0x7f591c0a6000 comp rx=0 tx=0).stop 2023-10-30T12:56:59.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 msgr2=0x7f59201f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:56:59.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5918007e60 tx=0x7f591800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 shutdown_connections 2023-10-30T12:56:59.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f59040664a0 0x7f5904068960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59201516c0 0x7f59201f22c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f59200c82d0 0x7f59201f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 --2- 172.21.15.8:0/2341085271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59200c2500 0x7f592006bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:56:59.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 >> 172.21.15.8:0/2341085271 conn(0x7f5920071fb0 msgr2=0x7f5920072390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:56:59.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 shutdown_connections 2023-10-30T12:56:59.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:56:59.346+0000 7f59275aa640 1 -- 172.21.15.8:0/2341085271 wait complete. 2023-10-30T12:56:59.373 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:57:01.374 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:01.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.522+0000 7fc5f453e640 1 Processor -- start 2023-10-30T12:57:01.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.522+0000 7fc5f453e640 1 -- start start 2023-10-30T12:57:01.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.522+0000 7fc5f453e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5ec0cb7d0 0x7fc5ec0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.522+0000 7fc5f453e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec0c2310 con 0x7fc5ec150830 2023-10-30T12:57:01.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f453e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec156710 con 0x7fc5ec0c3e40 2023-10-30T12:57:01.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f453e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec06bc10 con 0x7fc5ec0cb7d0 2023-10-30T12:57:01.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f22da640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fc5ec150830 0x7fc5ec06b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39380/0 (socket says 172.21.15.8:39380) 2023-10-30T12:57:01.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f22da640 1 -- 172.21.15.8:0/2815845286 learned_addr learned my addr 172.21.15.8:0/2815845286 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:01.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f1ad9640 1 --2- 172.21.15.8:0/2815845286 >> v2:172.21.15.46:3300/0 conn(0x7fc5ec0cb7d0 0x7fc5ec0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076482078 0 0) 0x7fc5ec0c2310 con 0x7fc5ec150830 2023-10-30T12:57:01.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d8005740 con 0x7fc5ec150830 2023-10-30T12:57:01.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022477 0 0) 0x7fc5d8005740 con 0x7fc5ec150830 2023-10-30T12:57:01.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 >> v2:172.21.15.46:3300/0 conn(0x7fc5ec0cb7d0 msgr2=0x7fc5ec0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:01.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 --2- 172.21.15.8:0/2815845286 >> v2:172.21.15.46:3300/0 conn(0x7fc5ec0cb7d0 0x7fc5ec0c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 >> v1:172.21.15.46:6789/0 conn(0x7fc5ec0c3e40 legacy=0x7fc5ec070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0c4360 con 0x7fc5ec150830 2023-10-30T12:57:01.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fc5e80016a0 con 0x7fc5ec150830 2023-10-30T12:57:01.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 >> v1:172.21.15.8:6789/0 conn(0x7fc5ec150830 legacy=0x7fc5ec06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d8005b90 0x7fc5d8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d80069e0 0x7fc5d8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 0x7fc5d800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec06bc10 con 0x7fc5d80073a0 2023-10-30T12:57:01.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d800e250 con 0x7fc5d80069e0 2023-10-30T12:57:01.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d800e410 con 0x7fc5d8005b90 2023-10-30T12:57:01.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 0x7fc5d800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f1ad9640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d80069e0 0x7fc5d8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f22da640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d8005b90 0x7fc5d8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d8005b90 msgr2=0x7fc5d8005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:01.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d8005b90 0x7fc5d8005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d80069e0 msgr2=0x7fc5d8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:01.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d80069e0 0x7fc5d8006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.526+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2815845286 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8005740 con 0x7fc5d80073a0 2023-10-30T12:57:01.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2815845286 reap_dead start 2023-10-30T12:57:01.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 0x7fc5d800dba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001e40 tx=0x7fc5e401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce9913d00d7ea98 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:01.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4003b80 con 0x7fc5d80073a0 2023-10-30T12:57:01.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4003ce0 con 0x7fc5d80073a0 2023-10-30T12:57:01.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f12d8640 1 -- 172.21.15.8:0/2815845286 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4005260 con 0x7fc5d80073a0 2023-10-30T12:57:01.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 msgr2=0x7fc5d800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 0x7fc5d800dba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001e40 tx=0x7fc5e401c740 comp rx=0 tx=0).stop 2023-10-30T12:57:01.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- 172.21.15.8:0/2815845286 shutdown_connections 2023-10-30T12:57:01.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2815845286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5d80073a0 0x7fc5d800dba0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- 172.21.15.8:0/2815845286 >> 172.21.15.8:0/2815845286 conn(0x7fc5ec0cac70 msgr2=0x7fc5ec0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:01.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- 172.21.15.8:0/2815845286 shutdown_connections 2023-10-30T12:57:01.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- 172.21.15.8:0/2815845286 wait complete. 2023-10-30T12:57:01.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 Processor -- start 2023-10-30T12:57:01.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.530+0000 7fc5f453e640 1 -- start start 2023-10-30T12:57:01.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d80073a0 0x7fc5ec0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5ec0c6020 0x7fc5ec14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec158180 con 0x7fc5ec1577c0 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec1596d0 con 0x7fc5ec0c6020 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec154750 con 0x7fc5d80073a0 2023-10-30T12:57:01.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39726/0 (socket says 172.21.15.8:39726) 2023-10-30T12:57:01.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2770593712 learned_addr learned my addr 172.21.15.8:0/2770593712 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:01.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f22da640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d80073a0 0x7fc5ec0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f1ad9640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5ec0c6020 0x7fc5ec14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d80073a0 msgr2=0x7fc5ec0ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d80073a0 0x7fc5ec0ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5ec0c6020 msgr2=0x7fc5ec14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5ec0c6020 0x7fc5ec14f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 -- 172.21.15.8:0/2770593712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec1f1bc0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f2adb640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401c570 tx=0x7fc5d8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4005430 con 0x7fc5ec1577c0 2023-10-30T12:57:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e40055b0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec1f1ea0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40a6440 con 0x7fc5ec1577c0 2023-10-30T12:57:01.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec1f2460 con 0x7fc5ec1577c0 2023-10-30T12:57:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec04e170 con 0x7fc5ec1577c0 2023-10-30T12:57:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.534+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc5e40a65a0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.538+0000 7fc5e2ffd640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 0x7fc5d0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.538+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fc5e411cef0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.538+0000 7fc5f22da640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 0x7fc5d0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.538+0000 7fc5f22da640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 0x7fc5d0068990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c10 tx=0x7fc5e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:01.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.542+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc5e40ea090 con 0x7fc5ec1577c0 2023-10-30T12:57:01.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.734+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5ec071480 con 0x7fc5ec1577c0 2023-10-30T12:57:01.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.734+0000 7fc5e2ffd640 1 -- 172.21.15.8:0/2770593712 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7fc5e40ed6b0 con 0x7fc5ec1577c0 2023-10-30T12:57:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:01.741 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:01.742 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:57:01.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 msgr2=0x7fc5d0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 0x7fc5d0068990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c10 tx=0x7fc5e8009040 comp rx=0 tx=0).stop 2023-10-30T12:57:01.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 msgr2=0x7fc5ec157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:01.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401c570 tx=0x7fc5d8007a10 comp rx=0 tx=0).stop 2023-10-30T12:57:01.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 shutdown_connections 2023-10-30T12:57:01.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc5d00664d0 0x7fc5d0068990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5ec0c6020 0x7fc5ec14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5ec1577c0 0x7fc5ec157c40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 --2- 172.21.15.8:0/2770593712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc5d80073a0 0x7fc5ec0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 >> 172.21.15.8:0/2770593712 conn(0x7fc5ec0cac70 msgr2=0x7fc5ec073740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:01.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 shutdown_connections 2023-10-30T12:57:01.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:01.738+0000 7fc5f453e640 1 -- 172.21.15.8:0/2770593712 wait complete. 2023-10-30T12:57:01.762 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:57:03.763 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:03.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 Processor -- start 2023-10-30T12:57:03.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 -- start start 2023-10-30T12:57:03.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c4a10 0x7ff4f815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f8159f20 con 0x7ff4f8153c80 2023-10-30T12:57:03.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f815dec0 con 0x7ff4f80c42a0 2023-10-30T12:57:03.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff500068640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81599a0 con 0x7ff4f80c4a10 2023-10-30T12:57:03.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39730/0 (socket says 172.21.15.8:39730) 2023-10-30T12:57:03.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 -- 172.21.15.8:0/3359457027 learned_addr learned my addr 172.21.15.8:0/3359457027 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:03.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fd603640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c4a10 0x7ff4f815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 -- 172.21.15.8:0/3359457027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c4a10 msgr2=0x7ff4f815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:03.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c4a10 0x7ff4f815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 -- 172.21.15.8:0/3359457027 >> v1:172.21.15.46:6789/0 conn(0x7ff4f8153c80 legacy=0x7ff4f815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:03.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.914+0000 7ff4fe605640 1 -- 172.21.15.8:0/3359457027 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f815d620 con 0x7ff4f80c42a0 2023-10-30T12:57:03.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff4fe605640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009e50 tx=0x7ff4f001f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50641b32770f154 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:03.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff4fce02640 1 -- 172.21.15.8:0/3359457027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00af400 con 0x7ff4f80c42a0 2023-10-30T12:57:03.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff4fce02640 1 -- 172.21.15.8:0/3359457027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f00ab050 con 0x7ff4f80c42a0 2023-10-30T12:57:03.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff4fce02640 1 -- 172.21.15.8:0/3359457027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00af780 con 0x7ff4f80c42a0 2023-10-30T12:57:03.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 -- 172.21.15.8:0/3359457027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80c42a0 msgr2=0x7ff4f80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:03.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009e50 tx=0x7ff4f001f1f0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 -- 172.21.15.8:0/3359457027 shutdown_connections 2023-10-30T12:57:03.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80c42a0 0x7ff4f80ca1e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 --2- 172.21.15.8:0/3359457027 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c4a10 0x7ff4f815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 -- 172.21.15.8:0/3359457027 >> 172.21.15.8:0/3359457027 conn(0x7ff4f80715c0 msgr2=0x7ff4f80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:03.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 -- 172.21.15.8:0/3359457027 shutdown_connections 2023-10-30T12:57:03.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.918+0000 7ff500068640 1 -- 172.21.15.8:0/3359457027 wait complete. 2023-10-30T12:57:03.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 Processor -- start 2023-10-30T12:57:03.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- start start 2023-10-30T12:57:03.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c42a0 0x7ff4f80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f80c4a10 0x7ff4f81f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80cfb80 con 0x7ff4f8153c80 2023-10-30T12:57:03.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81f2bb0 con 0x7ff4f80c4a10 2023-10-30T12:57:03.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80cfdc0 con 0x7ff4f80c42a0 2023-10-30T12:57:03.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39734/0 (socket says 172.21.15.8:39734) 2023-10-30T12:57:03.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 -- 172.21.15.8:0/471242901 learned_addr learned my addr 172.21.15.8:0/471242901 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:03.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fde04640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c42a0 0x7ff4f80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fd603640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f80c4a10 0x7ff4f81f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 -- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c42a0 msgr2=0x7ff4f80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:03.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c42a0 0x7ff4f80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 -- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f80c4a10 msgr2=0x7ff4f81f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:03.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f80c4a10 0x7ff4f81f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:03.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 -- 172.21.15.8:0/471242901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009b30 con 0x7ff4f8153c80 2023-10-30T12:57:03.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4fe605640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff4f001ce20 tx=0x7ff4f001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:03.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00afde0 con 0x7ff4f8153c80 2023-10-30T12:57:03.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0009510 con 0x7ff4f8153c80 2023-10-30T12:57:03.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00ba660 con 0x7ff4f8153c80 2023-10-30T12:57:03.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f81ee890 con 0x7ff4f8153c80 2023-10-30T12:57:03.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.922+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81eee50 con 0x7ff4f8153c80 2023-10-30T12:57:03.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f804e170 con 0x7ff4f8153c80 2023-10-30T12:57:03.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4f00b6040 con 0x7ff4f8153c80 2023-10-30T12:57:03.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff4ee7fc640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 0x7ff4dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:03.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7ff4f0122430 con 0x7ff4f8153c80 2023-10-30T12:57:03.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff4fde04640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 0x7ff4dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:03.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.926+0000 7ff4fde04640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 0x7ff4dc068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4000f70 tx=0x7ff4f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:03.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:03.930+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4f00ef600 con 0x7ff4f8153c80 2023-10-30T12:57:04.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.118+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4f80cad40 con 0x7ff4f8153c80 2023-10-30T12:57:04.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.118+0000 7ff4ee7fc640 1 -- 172.21.15.8:0/471242901 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7ff4f00f2c20 con 0x7ff4f8153c80 2023-10-30T12:57:04.126 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:04.126 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:04.126 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:57:04.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 msgr2=0x7ff4dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:04.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 0x7ff4dc068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4000f70 tx=0x7ff4f4009040 comp rx=0 tx=0).stop 2023-10-30T12:57:04.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 msgr2=0x7ff4f8151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:04.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff4f001ce20 tx=0x7ff4f001bf90 comp rx=0 tx=0).stop 2023-10-30T12:57:04.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 shutdown_connections 2023-10-30T12:57:04.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff4dc0664a0 0x7ff4dc068960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:04.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f80c4a10 0x7ff4f81f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:04.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f8153c80 0x7ff4f8151710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:04.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 --2- 172.21.15.8:0/471242901 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff4f80c42a0 0x7ff4f80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:04.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 >> 172.21.15.8:0/471242901 conn(0x7ff4f80715c0 msgr2=0x7ff4f8152580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:04.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 shutdown_connections 2023-10-30T12:57:04.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:04.122+0000 7ff500068640 1 -- 172.21.15.8:0/471242901 wait complete. 2023-10-30T12:57:04.148 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:57:06.149 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:06.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 Processor -- start 2023-10-30T12:57:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 -- start start 2023-10-30T12:57:06.300 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f779015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7790159f20 con 0x7f7790153c80 2023-10-30T12:57:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779015d620 con 0x7f77900c42a0 2023-10-30T12:57:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f779586b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779015dec0 con 0x7f77900c4a10 2023-10-30T12:57:06.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7790153c80 0x7f779015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39384/0 (socket says 172.21.15.8:39384) 2023-10-30T12:57:06.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778effd640 1 -- 172.21.15.8:0/147891305 learned_addr learned my addr 172.21.15.8:0/147891305 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f7787fff640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f779015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250804424 0 0) 0x7f7790159f20 con 0x7f7790153c80 2023-10-30T12:57:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f777c003530 con 0x7f7790153c80 2023-10-30T12:57:06.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263023044 0 0) 0x7f777c003530 con 0x7f7790153c80 2023-10-30T12:57:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 msgr2=0x7f779015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f779015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 >> v1:172.21.15.46:6789/0 conn(0x7f77900c42a0 legacy=0x7f77900ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779015d900 con 0x7f7790153c80 2023-10-30T12:57:06.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f7780001680 con 0x7f7790153c80 2023-10-30T12:57:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 >> v1:172.21.15.8:6789/0 conn(0x7f7790153c80 legacy=0x7f779015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f777c0039b0 0x7f777c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f777c004360 0x7f777c004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f777c005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779015dec0 con 0x7f777c0039b0 2023-10-30T12:57:06.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.294+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c00b990 con 0x7f777c004360 2023-10-30T12:57:06.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c00bb50 con 0x7f77900c4a10 2023-10-30T12:57:06.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f777c0039b0 0x7f777c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.307 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778f7fe640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f777c005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 -- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 msgr2=0x7f777c005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:06.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77900c4a10 0x7f777c005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 -- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f777c004360 msgr2=0x7f777c004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:06.308 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f7787fff640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f777c004360 0x7f777c004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f777c004360 0x7f777c004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 -- 172.21.15.8:0/147891305 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c003530 con 0x7f777c0039b0 2023-10-30T12:57:06.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f7787fff640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f777c004360 0x7f777c004830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:57:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778effd640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f777c0039b0 0x7f777c003d90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f778000c190 tx=0x7f778001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf226c70dd73fce9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:06.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780005ae0 con 0x7f777c0039b0 2023-10-30T12:57:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7780005c60 con 0x7f777c0039b0 2023-10-30T12:57:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.298+0000 7f778e7fc640 1 -- 172.21.15.8:0/147891305 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800a58e0 con 0x7f777c0039b0 2023-10-30T12:57:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- 172.21.15.8:0/147891305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f777c0039b0 msgr2=0x7f777c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 --2- 172.21.15.8:0/147891305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f777c0039b0 0x7f777c003d90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f778000c190 tx=0x7f778001cbe0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f778f7fe640 1 -- 172.21.15.8:0/147891305 reap_dead start 2023-10-30T12:57:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- 172.21.15.8:0/147891305 shutdown_connections 2023-10-30T12:57:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- 172.21.15.8:0/147891305 >> 172.21.15.8:0/147891305 conn(0x7f77900715c0 msgr2=0x7f77900719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- 172.21.15.8:0/147891305 shutdown_connections 2023-10-30T12:57:06.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- 172.21.15.8:0/147891305 wait complete. 2023-10-30T12:57:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 Processor -- start 2023-10-30T12:57:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- start start 2023-10-30T12:57:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f777c0039b0 0x7f77901eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 0x7f77900c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790157d40 con 0x7f77901f2050 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77901f2590 con 0x7f77900c58b0 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f779586b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77901f2870 con 0x7f777c0039b0 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39764/0 (socket says 172.21.15.8:39764) 2023-10-30T12:57:06.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 -- 172.21.15.8:0/829663824 learned_addr learned my addr 172.21.15.8:0/829663824 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:06.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f778effd640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f777c0039b0 0x7f77901eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f778f7fe640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 0x7f77900c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 -- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f777c0039b0 msgr2=0x7f77901eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f777c0039b0 0x7f77901eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 -- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 msgr2=0x7f77900c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 0x7f77900c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f7787fff640 1 -- 172.21.15.8:0/829663824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77901f2b50 con 0x7f77901f2050 2023-10-30T12:57:06.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.302+0000 7f778f7fe640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 0x7f77900c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:06.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f7787fff640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7778009fd0 tx=0x7f7778010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:06.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778004220 con 0x7f77901f2050 2023-10-30T12:57:06.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778004380 con 0x7f77901f2050 2023-10-30T12:57:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777800eb60 con 0x7f77901f2050 2023-10-30T12:57:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77901f2e30 con 0x7f77901f2050 2023-10-30T12:57:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790149590 con 0x7f77901f2050 2023-10-30T12:57:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790159b40 con 0x7f77901f2050 2023-10-30T12:57:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f777800ecc0 con 0x7f77901f2050 2023-10-30T12:57:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 0x7f7770068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f7778083e60 con 0x7f77901f2050 2023-10-30T12:57:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.306+0000 7f778effd640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 0x7f7770068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.310+0000 7f778effd640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 0x7f7770068990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7780005fd0 tx=0x7f77800a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.310+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7778051140 con 0x7f77901f2050 2023-10-30T12:57:06.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f779004e170 con 0x7f77901f2050 2023-10-30T12:57:06.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f778cff9640 1 -- 172.21.15.8:0/829663824 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6523 (secure 0 0 0) 0x7f7778054760 con 0x7f77901f2050 2023-10-30T12:57:06.512 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:06.512 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:56:25.375789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5079,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5079":{"gid":5079,"name":"e","rank":1,"incarnation":31,"state":"up:active","state_seq":26,"addr":"172.21.15.8:6838/3033347051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":3033347051},{"type":"v1","addr":"172.21.15.8:6838","nonce":3033347051}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:06.514 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 59 2023-10-30T12:57:06.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 msgr2=0x7f7770068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 0x7f7770068990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7780005fd0 tx=0x7f77800a7070 comp rx=0 tx=0).stop 2023-10-30T12:57:06.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 msgr2=0x7f7790149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:06.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7778009fd0 tx=0x7f7778010d60 comp rx=0 tx=0).stop 2023-10-30T12:57:06.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 shutdown_connections 2023-10-30T12:57:06.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f77700664d0 0x7f7770068990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77900c58b0 0x7f77900c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77901f2050 0x7f7790149050 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 --2- 172.21.15.8:0/829663824 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f777c0039b0 0x7f77901eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:06.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 >> 172.21.15.8:0/829663824 conn(0x7f77900715c0 msgr2=0x7f7790152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:06.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 shutdown_connections 2023-10-30T12:57:06.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:06.506+0000 7f779586b640 1 -- 172.21.15.8:0/829663824 wait complete. 2023-10-30T12:57:06.532 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-30T12:57:08.533 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:08.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 Processor -- start 2023-10-30T12:57:08.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 -- start start 2023-10-30T12:57:08.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f169c15cf90 0x7f169c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c159f20 con 0x7f169c153c80 2023-10-30T12:57:08.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c0c47e0 con 0x7f169c0c4a10 2023-10-30T12:57:08.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a432a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169c15dec0 con 0x7f169c15cf90 2023-10-30T12:57:08.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a20c6640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f169c153c80 0x7f169c15a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:39394/0 (socket says 172.21.15.8:39394) 2023-10-30T12:57:08.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a20c6640 1 -- 172.21.15.8:0/534616712 learned_addr learned my addr 172.21.15.8:0/534616712 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:08.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a28c7640 1 --2- 172.21.15.8:0/534616712 >> v2:172.21.15.46:3300/0 conn(0x7f169c15cf90 0x7f169c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960084760 0 0) 0x7f169c159f20 con 0x7f169c153c80 2023-10-30T12:57:08.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1688003530 con 0x7f169c153c80 2023-10-30T12:57:08.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851622553 0 0) 0x7f1688003530 con 0x7f169c153c80 2023-10-30T12:57:08.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 >> v2:172.21.15.46:3300/0 conn(0x7f169c15cf90 msgr2=0x7f169c0c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:08.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 --2- 172.21.15.8:0/534616712 >> v2:172.21.15.46:3300/0 conn(0x7f169c15cf90 0x7f169c0c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 >> v1:172.21.15.46:6789/0 conn(0x7f169c0c4a10 legacy=0x7f169c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c15d620 con 0x7f169c153c80 2023-10-30T12:57:08.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f1698001680 con 0x7f169c153c80 2023-10-30T12:57:08.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 >> v1:172.21.15.8:6789/0 conn(0x7f169c153c80 legacy=0x7f169c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1688003980 0x7f1688003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 0x7f1688004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1688005190 0x7f168800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c15dec0 con 0x7f16880047d0 2023-10-30T12:57:08.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168800c040 con 0x7f1688005190 2023-10-30T12:57:08.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168800c200 con 0x7f1688003980 2023-10-30T12:57:08.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.682+0000 7f16a18c5640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 0x7f1688004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a20c6640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1688003980 0x7f1688003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a28c7640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1688005190 0x7f168800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 -- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1688003980 msgr2=0x7f1688003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1688003980 0x7f1688003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 -- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1688005190 msgr2=0x7f168800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1688005190 0x7f168800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 -- 172.21.15.8:0/534616712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688003530 con 0x7f16880047d0 2023-10-30T12:57:08.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a28c7640 1 -- 172.21.15.8:0/534616712 reap_dead start 2023-10-30T12:57:08.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a18c5640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 0x7f1688004c50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f168c001750 tx=0x7f168c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2773b1d39ba4558 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:08.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0052d0 con 0x7f16880047d0 2023-10-30T12:57:08.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c0a7050 con 0x7f16880047d0 2023-10-30T12:57:08.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a10c4640 1 -- 172.21.15.8:0/534616712 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c003600 con 0x7f16880047d0 2023-10-30T12:57:08.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a432a640 1 -- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 msgr2=0x7f1688004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.686+0000 7f16a432a640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 0x7f1688004c50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f168c001750 tx=0x7f168c01c3a0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- 172.21.15.8:0/534616712 shutdown_connections 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 --2- 172.21.15.8:0/534616712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16880047d0 0x7f1688004c50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- 172.21.15.8:0/534616712 >> 172.21.15.8:0/534616712 conn(0x7f169c0715c0 msgr2=0x7f169c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- 172.21.15.8:0/534616712 shutdown_connections 2023-10-30T12:57:08.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- 172.21.15.8:0/534616712 wait complete. 2023-10-30T12:57:08.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 Processor -- start 2023-10-30T12:57:08.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- start start 2023-10-30T12:57:08.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 0x7f169c14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 0x7f169c14d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c158130 con 0x7f169c0c8890 2023-10-30T12:57:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c157e50 con 0x7f169c14b8d0 2023-10-30T12:57:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a432a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c14c400 con 0x7f16880047d0 2023-10-30T12:57:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39784/0 (socket says 172.21.15.8:39784) 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 -- 172.21.15.8:0/1719368213 learned_addr learned my addr 172.21.15.8:0/1719368213 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a18c5640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 0x7f169c14d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a20c6640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 0x7f169c14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 -- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 msgr2=0x7f169c14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 0x7f169c14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 -- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 msgr2=0x7f169c14d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 0x7f169c14d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 -- 172.21.15.8:0/1719368213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c14c6e0 con 0x7f169c0c8890 2023-10-30T12:57:08.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a20c6640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 0x7f169c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:08.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a18c5640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 0x7f169c14d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:08.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.690+0000 7f16a28c7640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f169c0c97d0 tx=0x7f16940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:08.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400e070 con 0x7f169c0c8890 2023-10-30T12:57:08.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694011db0 con 0x7f169c0c8890 2023-10-30T12:57:08.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c14c9c0 con 0x7f169c0c8890 2023-10-30T12:57:08.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940039a0 con 0x7f169c0c8890 2023-10-30T12:57:08.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c150ab0 con 0x7f169c0c8890 2023-10-30T12:57:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c04e170 con 0x7f169c0c8890 2023-10-30T12:57:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1694003b00 con 0x7f169c0c8890 2023-10-30T12:57:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 0x7f1680068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f1694084b20 con 0x7f169c0c8890 2023-10-30T12:57:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.694+0000 7f16a20c6640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 0x7f1680068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.698+0000 7f16a20c6640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 0x7f1680068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1698001ca0 tx=0x7f1698009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.698+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1694051bd0 con 0x7f169c0c8890 2023-10-30T12:57:08.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f169c0cad40 con 0x7f169c0c8890 2023-10-30T12:57:08.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f1692ffd640 1 -- 172.21.15.8:0/1719368213 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f16940551f0 con 0x7f169c0c8890 2023-10-30T12:57:08.896 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:08.896 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:07.233420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:08.897 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 60 2023-10-30T12:57:08.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 msgr2=0x7f1680068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 0x7f1680068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1698001ca0 tx=0x7f1698009040 comp rx=0 tx=0).stop 2023-10-30T12:57:08.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 msgr2=0x7f169c149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:08.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f169c0c97d0 tx=0x7f16940115a0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 shutdown_connections 2023-10-30T12:57:08.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f16800664a0 0x7f1680068960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f169c14b8d0 0x7f169c14d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f169c0c8890 0x7f169c149b30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 --2- 172.21.15.8:0/1719368213 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f16880047d0 0x7f169c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:08.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 >> 172.21.15.8:0/1719368213 conn(0x7f169c0715c0 msgr2=0x7f169c151040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:08.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 shutdown_connections 2023-10-30T12:57:08.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:08.890+0000 7f16a432a640 1 -- 172.21.15.8:0/1719368213 wait complete. 2023-10-30T12:57:08.917 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-30T12:57:08.918 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-30T12:57:08.918 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.058+0000 7febd012a640 1 Processor -- start 2023-10-30T12:57:09.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.058+0000 7febd012a640 1 -- start start 2023-10-30T12:57:09.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febd012a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febc806c420 0x7febc80c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febd012a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7febc80c7170 0x7febc8159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febd012a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80c5f80 con 0x7febc8153bb0 2023-10-30T12:57:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febd012a640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80c7580 con 0x7febc80c7170 2023-10-30T12:57:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febd012a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80c5690 con 0x7febc806c420 2023-10-30T12:57:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7febc80c7170 0x7febc8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7febc80c7170 0x7febc8159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39788/0 (socket says 172.21.15.8:39788) 2023-10-30T12:57:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 -- 172.21.15.8:0/1450374904 learned_addr learned my addr 172.21.15.8:0/1450374904 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febcd6c5640 1 --2- 172.21.15.8:0/1450374904 >> v2:172.21.15.46:3300/0 conn(0x7febc806c420 0x7febc80c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 -- 172.21.15.8:0/1450374904 >> v2:172.21.15.46:3300/0 conn(0x7febc806c420 msgr2=0x7febc80c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 --2- 172.21.15.8:0/1450374904 >> v2:172.21.15.46:3300/0 conn(0x7febc806c420 0x7febc80c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 -- 172.21.15.8:0/1450374904 >> v1:172.21.15.46:6789/0 conn(0x7febc8153bb0 legacy=0x7febc80c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.062+0000 7febce6c7640 1 -- 172.21.15.8:0/1450374904 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc815a580 con 0x7febc80c7170 2023-10-30T12:57:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febce6c7640 1 --2- 172.21.15.8:0/1450374904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc80c7170 0x7febc8159980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7febc806f800 tx=0x7febc001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40daf44e4a4fd1ab server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febccec4640 1 -- 172.21.15.8:0/1450374904 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00af400 con 0x7febc80c7170 2023-10-30T12:57:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febccec4640 1 -- 172.21.15.8:0/1450374904 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc00ab050 con 0x7febc80c7170 2023-10-30T12:57:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febccec4640 1 -- 172.21.15.8:0/1450374904 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00af780 con 0x7febc80c7170 2023-10-30T12:57:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- 172.21.15.8:0/1450374904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc80c7170 msgr2=0x7febc8159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 --2- 172.21.15.8:0/1450374904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc80c7170 0x7febc8159980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7febc806f800 tx=0x7febc001f1c0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- 172.21.15.8:0/1450374904 shutdown_connections 2023-10-30T12:57:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 --2- 172.21.15.8:0/1450374904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc80c7170 0x7febc8159980 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 --2- 172.21.15.8:0/1450374904 >> v2:172.21.15.46:3300/0 conn(0x7febc806c420 0x7febc80c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- 172.21.15.8:0/1450374904 >> 172.21.15.8:0/1450374904 conn(0x7febc80cac70 msgr2=0x7febc80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- 172.21.15.8:0/1450374904 shutdown_connections 2023-10-30T12:57:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- 172.21.15.8:0/1450374904 wait complete. 2023-10-30T12:57:09.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 Processor -- start 2023-10-30T12:57:09.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.066+0000 7febd012a640 1 -- start start 2023-10-30T12:57:09.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc806c420 0x7febc81ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7febc80c7170 0x7febc80c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81ef6c0 con 0x7febc8153bb0 2023-10-30T12:57:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8070ac0 con 0x7febc806c420 2023-10-30T12:57:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc80bf800 con 0x7febc80c7170 2023-10-30T12:57:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39804/0 (socket says 172.21.15.8:39804) 2023-10-30T12:57:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 -- 172.21.15.8:0/2853687611 learned_addr learned my addr 172.21.15.8:0/2853687611 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febcd6c5640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7febc80c7170 0x7febc80c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febcdec6640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc806c420 0x7febc81ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 -- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7febc80c7170 msgr2=0x7febc80c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7febc80c7170 0x7febc80c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 -- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc806c420 msgr2=0x7febc81ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc806c420 0x7febc81ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 -- 172.21.15.8:0/2853687611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009b30 con 0x7febc8153bb0 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febce6c7640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febc00b2040 tx=0x7febc001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00afd60 con 0x7febc8153bb0 2023-10-30T12:57:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0009510 con 0x7febc8153bb0 2023-10-30T12:57:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00ba660 con 0x7febc8153bb0 2023-10-30T12:57:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc81f1cb0 con 0x7febc8153bb0 2023-10-30T12:57:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81f2270 con 0x7febc8153bb0 2023-10-30T12:57:09.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc804e170 con 0x7febc8153bb0 2023-10-30T12:57:09.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.070+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7febc00b6040 con 0x7febc8153bb0 2023-10-30T12:57:09.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.074+0000 7febbe7fc640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 0x7febac068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:09.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.074+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7febc01224f0 con 0x7febc8153bb0 2023-10-30T12:57:09.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.074+0000 7febcdec6640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 0x7febac068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:09.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.074+0000 7febcdec6640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 0x7febac068990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f70 tx=0x7febc4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:09.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.078+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7febc00ef6c0 con 0x7febc8153bb0 2023-10-30T12:57:09.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc8066510 con 0x7febc8153bb0 2023-10-30T12:57:09.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febbe7fc640 1 -- 172.21.15.8:0/2853687611 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5731 (secure 0 0 0) 0x7febc00f2ce0 con 0x7febc8153bb0 2023-10-30T12:57:09.272 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:09.273 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:07.233420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:09.274 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 60 2023-10-30T12:57:09.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 msgr2=0x7febac068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 0x7febac068990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f70 tx=0x7febc4009040 comp rx=0 tx=0).stop 2023-10-30T12:57:09.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 msgr2=0x7febc80bfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:09.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febc00b2040 tx=0x7febc001bf60 comp rx=0 tx=0).stop 2023-10-30T12:57:09.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 shutdown_connections 2023-10-30T12:57:09.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7febac0664d0 0x7febac068990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febc8153bb0 0x7febc80bfb10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7febc80c7170 0x7febc80c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 --2- 172.21.15.8:0/2853687611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc806c420 0x7febc81ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:09.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 >> 172.21.15.8:0/2853687611 conn(0x7febc80cac70 msgr2=0x7febc8070170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:09.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 shutdown_connections 2023-10-30T12:57:09.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:09.266+0000 7febd012a640 1 -- 172.21.15.8:0/2853687611 wait complete. 2023-10-30T12:57:09.294 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 60, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 60, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 31 }, "gid_5394": { "addr": "172.21.15.8:6837/1849048959", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 1849048959, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 1849048959, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5394, "incarnation": 52, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5535": { "addr": "172.21.15.8:6839/4033248661", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 4033248661, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 4033248661, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5535, "incarnation": 60, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:57:07.233420+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5535, "mds_2": 5394 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:57:11.295 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:11.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 Processor -- start 2023-10-30T12:57:11.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 -- start start 2023-10-30T12:57:11.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f587406b760 0x7f587415e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f5874157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f587406c420 0x7f5874157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5874071e00 con 0x7f587406b760 2023-10-30T12:57:11.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5874070020 con 0x7f587406c420 2023-10-30T12:57:11.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f587a73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587415eb90 con 0x7f587406bb40 2023-10-30T12:57:11.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f587406b760 0x7f587415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f587406b760 0x7f587415e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53800/0 (socket says 172.21.15.8:53800) 2023-10-30T12:57:11.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 -- 172.21.15.8:0/3272290756 learned_addr learned my addr 172.21.15.8:0/3272290756 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:11.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f58737fe640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f5874157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5878cdb640 1 --2- 172.21.15.8:0/3272290756 >> v2:172.21.15.46:3300/0 conn(0x7f587406c420 0x7f5874157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 -- 172.21.15.8:0/3272290756 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 msgr2=0x7f5874157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:11.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f5874157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 -- 172.21.15.8:0/3272290756 >> v2:172.21.15.46:3300/0 conn(0x7f587406c420 msgr2=0x7f5874157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:11.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 --2- 172.21.15.8:0/3272290756 >> v2:172.21.15.46:3300/0 conn(0x7f587406c420 0x7f5874157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.498+0000 7f5873fff640 1 -- 172.21.15.8:0/3272290756 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5874070f10 con 0x7f587406b760 2023-10-30T12:57:11.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f5873fff640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406b760 0x7f587415e650 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f587415b7a0 tx=0x7f586401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f206d5fb7582fbe server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:11.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f5872ffd640 1 -- 172.21.15.8:0/3272290756 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640aa5f0 con 0x7f587406b760 2023-10-30T12:57:11.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f5872ffd640 1 -- 172.21.15.8:0/3272290756 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640aa750 con 0x7f587406b760 2023-10-30T12:57:11.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f5872ffd640 1 -- 172.21.15.8:0/3272290756 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58640aaae0 con 0x7f587406b760 2023-10-30T12:57:11.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 -- 172.21.15.8:0/3272290756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406b760 msgr2=0x7f587415e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:11.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406b760 0x7f587415e650 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f587415b7a0 tx=0x7f586401c4d0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 -- 172.21.15.8:0/3272290756 shutdown_connections 2023-10-30T12:57:11.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 --2- 172.21.15.8:0/3272290756 >> v2:172.21.15.46:3300/0 conn(0x7f587406c420 0x7f5874157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f5874157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 --2- 172.21.15.8:0/3272290756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406b760 0x7f587415e650 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 -- 172.21.15.8:0/3272290756 >> 172.21.15.8:0/3272290756 conn(0x7f58740cac70 msgr2=0x7f58740cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:11.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 -- 172.21.15.8:0/3272290756 shutdown_connections 2023-10-30T12:57:11.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.502+0000 7f587a73e640 1 -- 172.21.15.8:0/3272290756 wait complete. 2023-10-30T12:57:11.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 Processor -- start 2023-10-30T12:57:11.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- start start 2023-10-30T12:57:11.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f587406b760 0x7f5874159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f587415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740c4c80 con 0x7f587406c420 2023-10-30T12:57:11.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58741f0400 con 0x7f587406b760 2023-10-30T12:57:11.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740c4730 con 0x7f587406bb40 2023-10-30T12:57:11.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53808/0 (socket says 172.21.15.8:53808) 2023-10-30T12:57:11.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 -- 172.21.15.8:0/960259431 learned_addr learned my addr 172.21.15.8:0/960259431 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:11.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5873fff640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f587406b760 0x7f5874159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f58737fe640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f587415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 -- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 msgr2=0x7f587415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:11.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f587415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 -- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f587406b760 msgr2=0x7f5874159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:11.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f587406b760 0x7f5874159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 -- 172.21.15.8:0/960259431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640094d0 con 0x7f587406c420 2023-10-30T12:57:11.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5878cdb640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f586c00efc0 tx=0x7f586c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:11.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c009ba0 con 0x7f587406c420 2023-10-30T12:57:11.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c00f040 con 0x7f587406c420 2023-10-30T12:57:11.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740c49a0 con 0x7f587406c420 2023-10-30T12:57:11.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0146c0 con 0x7f587406c420 2023-10-30T12:57:11.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.506+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58741f0090 con 0x7f587406c420 2023-10-30T12:57:11.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5874158ac0 con 0x7f587406c420 2023-10-30T12:57:11.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f586c014820 con 0x7f587406c420 2023-10-30T12:57:11.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f5870ff9640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:11.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f586c084e60 con 0x7f587406c420 2023-10-30T12:57:11.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f5873fff640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:11.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.510+0000 7f5873fff640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f58640ae000 tx=0x7f58640b5040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:11.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.514+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5874158ac0 con 0x7f587406c420 2023-10-30T12:57:11.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.706+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5874158ac0 con 0x7f587406c420 2023-10-30T12:57:11.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.706+0000 7f5870ff9640 1 -- 172.21.15.8:0/960259431 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f5874158ac0 con 0x7f587406c420 2023-10-30T12:57:11.714 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:11.714 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:07.233420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:11.715 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 60 2023-10-30T12:57:11.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 msgr2=0x7f5860068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:11.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f58640ae000 tx=0x7f58640b5040 comp rx=0 tx=0).stop 2023-10-30T12:57:11.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 msgr2=0x7f5874158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:11.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f586c00efc0 tx=0x7f586c00c490 comp rx=0 tx=0).stop 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 shutdown_connections 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f587406c420 0x7f5874158da0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f587406bb40 0x7f587415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 --2- 172.21.15.8:0/960259431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f587406b760 0x7f5874159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:11.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 >> 172.21.15.8:0/960259431 conn(0x7f58740cac70 msgr2=0x7f58740736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:11.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 shutdown_connections 2023-10-30T12:57:11.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:11.710+0000 7f587a73e640 1 -- 172.21.15.8:0/960259431 wait complete. 2023-10-30T12:57:11.735 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:13.736 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:13.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 Processor -- start 2023-10-30T12:57:13.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 -- start start 2023-10-30T12:57:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f80151c20 0x7f0f800c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0f800cb8b0 0x7f0f8006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f800c5030 con 0x7f0f80151840 2023-10-30T12:57:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f80157830 con 0x7f0f800cb8b0 2023-10-30T12:57:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f849d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80157b10 con 0x7f0f80151c20 2023-10-30T12:57:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0f800cb8b0 0x7f0f8006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f0f800cb8b0 0x7f0f8006b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53816/0 (socket says 172.21.15.8:53816) 2023-10-30T12:57:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 -- 172.21.15.8:0/1779163433 learned_addr learned my addr 172.21.15.8:0/1779163433 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7e59c640 1 --1- 172.21.15.8:0/1779163433 >> v1:172.21.15.8:6789/0 conn(0x7f0f80151840 0x7f0f80072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:46808/0 (socket says 172.21.15.8:46808) 2023-10-30T12:57:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7dd9b640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f80151c20 0x7f0f800c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 -- 172.21.15.8:0/1779163433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f80151c20 msgr2=0x7f0f800c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f80151c20 0x7f0f800c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 -- 172.21.15.8:0/1779163433 >> v1:172.21.15.8:6789/0 conn(0x7f0f80151840 legacy=0x7f0f80072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7ed9d640 1 -- 172.21.15.8:0/1779163433 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80157df0 con 0x7f0f800cb8b0 2023-10-30T12:57:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.874+0000 7f0f7d59a640 1 -- 172.21.15.8:0/1779163433 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547088199 0 0) 0x7f0f800c5030 con 0x7f0f80151840 2023-10-30T12:57:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f7ed9d640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f800cb8b0 0x7f0f8006b600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009e50 tx=0x7f0f7801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517444778aa64b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f7d59a640 1 -- 172.21.15.8:0/1779163433 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780af400 con 0x7f0f800cb8b0 2023-10-30T12:57:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f7d59a640 1 -- 172.21.15.8:0/1779163433 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f780ab050 con 0x7f0f800cb8b0 2023-10-30T12:57:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f7d59a640 1 -- 172.21.15.8:0/1779163433 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780af780 con 0x7f0f800cb8b0 2023-10-30T12:57:13.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 -- 172.21.15.8:0/1779163433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f800cb8b0 msgr2=0x7f0f8006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:13.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f800cb8b0 0x7f0f8006b600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009e50 tx=0x7f0f7801f1f0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 -- 172.21.15.8:0/1779163433 shutdown_connections 2023-10-30T12:57:13.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f800cb8b0 0x7f0f8006b600 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 --2- 172.21.15.8:0/1779163433 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f80151c20 0x7f0f800c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 -- 172.21.15.8:0/1779163433 >> 172.21.15.8:0/1779163433 conn(0x7f0f800cac70 msgr2=0x7f0f800cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 -- 172.21.15.8:0/1779163433 shutdown_connections 2023-10-30T12:57:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.878+0000 7f0f849d3640 1 -- 172.21.15.8:0/1779163433 wait complete. 2023-10-30T12:57:13.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 Processor -- start 2023-10-30T12:57:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- start start 2023-10-30T12:57:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f800cb8b0 0x7f0f80156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 0x7f0f801e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801532e0 con 0x7f0f80151840 2023-10-30T12:57:13.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8006fdf0 con 0x7f0f80151c20 2023-10-30T12:57:13.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80152f90 con 0x7f0f800cb8b0 2023-10-30T12:57:13.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53818/0 (socket says 172.21.15.8:53818) 2023-10-30T12:57:13.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 -- 172.21.15.8:0/3164388437 learned_addr learned my addr 172.21.15.8:0/3164388437 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:13.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7e59c640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f800cb8b0 0x7f0f80156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7ed9d640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 0x7f0f801e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 -- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f800cb8b0 msgr2=0x7f0f80156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:13.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f800cb8b0 0x7f0f80156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 -- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 msgr2=0x7f0f801e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 0x7f0f801e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 -- 172.21.15.8:0/3164388437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f78009b30 con 0x7f0f80151840 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7ed9d640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 0x7f0f801e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f7dd9b640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d7d0 tx=0x7f0f7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400b6c0 con 0x7f0f80151840 2023-10-30T12:57:13.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7400b820 con 0x7f0f80151840 2023-10-30T12:57:13.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400bbb0 con 0x7f0f80151840 2023-10-30T12:57:13.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f80154e60 con 0x7f0f80151840 2023-10-30T12:57:13.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.882+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801f2c20 con 0x7f0f80151840 2023-10-30T12:57:13.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8004e170 con 0x7f0f80151840 2023-10-30T12:57:13.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f7400bd10 con 0x7f0f80151840 2023-10-30T12:57:13.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f6effd640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 0x7f0f60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:13.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f0f74084710 con 0x7f0f80151840 2023-10-30T12:57:13.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f7e59c640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 0x7f0f60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:13.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.886+0000 7f0f7e59c640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 0x7f0f60068960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000f70 tx=0x7f0f70007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:13.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:13.890+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f740517d0 con 0x7f0f80151840 2023-10-30T12:57:14.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f8014f480 con 0x7f0f80151840 2023-10-30T12:57:14.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f6effd640 1 -- 172.21.15.8:0/3164388437 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f0f74054df0 con 0x7f0f80151840 2023-10-30T12:57:14.088 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:14.088 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:13.613813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:resolve","state_seq":25,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:14.089 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 61 2023-10-30T12:57:14.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 msgr2=0x7f0f60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:14.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 0x7f0f60068960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000f70 tx=0x7f0f70007b60 comp rx=0 tx=0).stop 2023-10-30T12:57:14.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 msgr2=0x7f0f80153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:14.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d7d0 tx=0x7f0f7400dc90 comp rx=0 tx=0).stop 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.082+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 shutdown_connections 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0f600664a0 0x7f0f60068960 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f80151c20 0x7f0f801e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f80151840 0x7f0f80153700 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 --2- 172.21.15.8:0/3164388437 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0f800cb8b0 0x7f0f80156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:14.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 >> 172.21.15.8:0/3164388437 conn(0x7f0f800cac70 msgr2=0x7f0f800c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 shutdown_connections 2023-10-30T12:57:14.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:14.086+0000 7f0f849d3640 1 -- 172.21.15.8:0/3164388437 wait complete. 2023-10-30T12:57:14.109 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:16.110 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:16.111 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:16.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:16.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 Processor -- start 2023-10-30T12:57:16.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 -- start start 2023-10-30T12:57:16.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f23740cb180 0x7f23741597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:16.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237415a770 con 0x7f23740c62b0 2023-10-30T12:57:16.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740c9d40 con 0x7f2374153b50 2023-10-30T12:57:16.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2379274640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2374159cf0 con 0x7f23740cb180 2023-10-30T12:57:16.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f23737fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f23740c62b0 0x7f237406ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:46822/0 (socket says 172.21.15.8:46822) 2023-10-30T12:57:16.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f23737fe640 1 -- 172.21.15.8:0/3142137601 learned_addr learned my addr 172.21.15.8:0/3142137601 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:16.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2378a73640 1 --2- 172.21.15.8:0/3142137601 >> v2:172.21.15.8:3300/0 conn(0x7f23740cb180 0x7f23741597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:16.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2378a73640 1 -- 172.21.15.8:0/3142137601 >> v1:172.21.15.46:6789/0 conn(0x7f2374153b50 legacy=0x7f23740ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2378a73640 1 -- 172.21.15.8:0/3142137601 >> v1:172.21.15.8:6789/0 conn(0x7f23740c62b0 legacy=0x7f237406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.262+0000 7f2378a73640 1 -- 172.21.15.8:0/3142137601 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374153f30 con 0x7f23740cb180 2023-10-30T12:57:16.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2378a73640 1 --2- 172.21.15.8:0/3142137601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23740cb180 0x7f23741597b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f236c009d90 tx=0x7f236c01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ff97800468a64b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:16.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2372ffd640 1 -- 172.21.15.8:0/3142137601 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0af600 con 0x7f23740cb180 2023-10-30T12:57:16.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2372ffd640 1 -- 172.21.15.8:0/3142137601 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c0ab050 con 0x7f23740cb180 2023-10-30T12:57:16.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2372ffd640 1 -- 172.21.15.8:0/3142137601 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0af980 con 0x7f23740cb180 2023-10-30T12:57:16.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 -- 172.21.15.8:0/3142137601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23740cb180 msgr2=0x7f23741597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 --2- 172.21.15.8:0/3142137601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23740cb180 0x7f23741597b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f236c009d90 tx=0x7f236c01f130 comp rx=0 tx=0).stop 2023-10-30T12:57:16.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 -- 172.21.15.8:0/3142137601 shutdown_connections 2023-10-30T12:57:16.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 --2- 172.21.15.8:0/3142137601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23740cb180 0x7f23741597b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 -- 172.21.15.8:0/3142137601 >> 172.21.15.8:0/3142137601 conn(0x7f2374065a80 msgr2=0x7f2374065e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:16.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 -- 172.21.15.8:0/3142137601 shutdown_connections 2023-10-30T12:57:16.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 -- 172.21.15.8:0/3142137601 wait complete. 2023-10-30T12:57:16.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.266+0000 7f2379274640 1 Processor -- start 2023-10-30T12:57:16.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- start start 2023-10-30T12:57:16.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740c62b0 0x7f237414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:16.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23740cb180 0x7f2374157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c48b0 con 0x7f2374153b50 2023-10-30T12:57:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c77e0 con 0x7f23740c62b0 2023-10-30T12:57:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23741ef830 con 0x7f23740cb180 2023-10-30T12:57:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:16.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53862/0 (socket says 172.21.15.8:53862) 2023-10-30T12:57:16.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 -- 172.21.15.8:0/2352560840 learned_addr learned my addr 172.21.15.8:0/2352560840 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:16.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2373fff640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740c62b0 0x7f237414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:16.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f23737fe640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23740cb180 0x7f2374157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:16.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 -- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23740cb180 msgr2=0x7f2374157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23740cb180 0x7f2374157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 -- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740c62b0 msgr2=0x7f237414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740c62b0 0x7f237414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 -- 172.21.15.8:0/2352560840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009a70 con 0x7f2374153b50 2023-10-30T12:57:16.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2378a73640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b2040 tx=0x7f236c01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:16.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0b9610 con 0x7f2374153b50 2023-10-30T12:57:16.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c0b9770 con 0x7f2374153b50 2023-10-30T12:57:16.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0b9b20 con 0x7f2374153b50 2023-10-30T12:57:16.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740c7a80 con 0x7f2374153b50 2023-10-30T12:57:16.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741576c0 con 0x7f2374153b50 2023-10-30T12:57:16.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.270+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23741578b0 con 0x7f2374153b50 2023-10-30T12:57:16.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.274+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f236c0b6050 con 0x7f2374153b50 2023-10-30T12:57:16.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.274+0000 7f2370ff9640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 0x7f2360068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:16.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.274+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f236c1226d0 con 0x7f2374153b50 2023-10-30T12:57:16.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.274+0000 7f2373fff640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 0x7f2360068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:16.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.274+0000 7f2373fff640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 0x7f2360068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2364007fd0 tx=0x7f2364009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:16.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.278+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f236c0ef8d0 con 0x7f2374153b50 2023-10-30T12:57:16.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.538+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2374067540 con 0x7f2374153b50 2023-10-30T12:57:16.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.538+0000 7f2370ff9640 1 -- 172.21.15.8:0/2352560840 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f236c0f2ef0 con 0x7f2374153b50 2023-10-30T12:57:16.545 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:16.545 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:13.613813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:resolve","state_seq":25,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:16.545 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 61 2023-10-30T12:57:16.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 msgr2=0x7f2360068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 0x7f2360068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2364007fd0 tx=0x7f2364009040 comp rx=0 tx=0).stop 2023-10-30T12:57:16.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 msgr2=0x7f2374158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:16.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b2040 tx=0x7f236c01bed0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 shutdown_connections 2023-10-30T12:57:16.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f23600664d0 0x7f2360068990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2374153b50 0x7f2374158120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f23740cb180 0x7f2374157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 --2- 172.21.15.8:0/2352560840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23740c62b0 0x7f237414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:16.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 >> 172.21.15.8:0/2352560840 conn(0x7f2374065a80 msgr2=0x7f2374150080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:16.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 shutdown_connections 2023-10-30T12:57:16.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:16.542+0000 7f2379274640 1 -- 172.21.15.8:0/2352560840 wait complete. 2023-10-30T12:57:16.565 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:18.567 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:18.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 Processor -- start 2023-10-30T12:57:18.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 -- start start 2023-10-30T12:57:18.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f582806b760 0x7f582815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806bb40 0x7f5828157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5828071e00 con 0x7f582806c420 2023-10-30T12:57:18.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5828070020 con 0x7f582806b760 2023-10-30T12:57:18.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582fa9e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582815eb90 con 0x7f582806bb40 2023-10-30T12:57:18.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582d83a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f582806b760 0x7f582815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582d83a640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f582806b760 0x7f582815e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53864/0 (socket says 172.21.15.8:53864) 2023-10-30T12:57:18.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582d83a640 1 -- 172.21.15.8:0/3118469333 learned_addr learned my addr 172.21.15.8:0/3118469333 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:18.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582e03b640 1 --1- 172.21.15.8:0/3118469333 >> v1:172.21.15.8:6789/0 conn(0x7f582806c420 0x7f58280c57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:46836/0 (socket says 172.21.15.8:46836) 2023-10-30T12:57:18.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.714+0000 7f582d039640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806bb40 0x7f5828157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582d83a640 1 -- 172.21.15.8:0/3118469333 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806bb40 msgr2=0x7f5828157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:18.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582d83a640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806bb40 0x7f5828157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582d83a640 1 -- 172.21.15.8:0/3118469333 >> v1:172.21.15.8:6789/0 conn(0x7f582806c420 legacy=0x7f58280c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582d83a640 1 -- 172.21.15.8:0/3118469333 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828070f10 con 0x7f582806b760 2023-10-30T12:57:18.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582c838640 1 -- 172.21.15.8:0/3118469333 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248047091 0 0) 0x7f5828071e00 con 0x7f582806c420 2023-10-30T12:57:18.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582d83a640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806b760 0x7f582815e650 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5824009870 tx=0x7f582401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e147c3bcd96645 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:18.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582c838640 1 -- 172.21.15.8:0/3118469333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240aa670 con 0x7f582806b760 2023-10-30T12:57:18.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582c838640 1 -- 172.21.15.8:0/3118469333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240aa7d0 con 0x7f582806b760 2023-10-30T12:57:18.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582c838640 1 -- 172.21.15.8:0/3118469333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240aab60 con 0x7f582806b760 2023-10-30T12:57:18.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 -- 172.21.15.8:0/3118469333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806b760 msgr2=0x7f582815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806b760 0x7f582815e650 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5824009870 tx=0x7f582401c510 comp rx=0 tx=0).stop 2023-10-30T12:57:18.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 -- 172.21.15.8:0/3118469333 shutdown_connections 2023-10-30T12:57:18.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806bb40 0x7f5828157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 --2- 172.21.15.8:0/3118469333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806b760 0x7f582815e650 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 -- 172.21.15.8:0/3118469333 >> 172.21.15.8:0/3118469333 conn(0x7f58280cac70 msgr2=0x7f58280cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:18.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.718+0000 7f582fa9e640 1 -- 172.21.15.8:0/3118469333 shutdown_connections 2023-10-30T12:57:18.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 -- 172.21.15.8:0/3118469333 wait complete. 2023-10-30T12:57:18.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 Processor -- start 2023-10-30T12:57:18.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 -- start start 2023-10-30T12:57:18.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 0x7f5828149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 0x7f582814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582814f4f0 con 0x7f582806c420 2023-10-30T12:57:18.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582814df00 con 0x7f582806bb40 2023-10-30T12:57:18.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582fa9e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582814f210 con 0x7f582806b760 2023-10-30T12:57:18.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53868/0 (socket says 172.21.15.8:53868) 2023-10-30T12:57:18.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 -- 172.21.15.8:0/202666784 learned_addr learned my addr 172.21.15.8:0/202666784 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:18.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582d83a640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 0x7f5828149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582d039640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 0x7f582814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 -- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 msgr2=0x7f5828149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 0x7f5828149fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 -- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 msgr2=0x7f582814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 0x7f582814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 -- 172.21.15.8:0/202666784 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240094d0 con 0x7f582806c420 2023-10-30T12:57:18.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582d039640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 0x7f582814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:18.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582d83a640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 0x7f5828149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:18.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f582e03b640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f582000c7e0 tx=0x7f582000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:18.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000a640 con 0x7f582806c420 2023-10-30T12:57:18.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582000a7a0 con 0x7f582806c420 2023-10-30T12:57:18.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.722+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ab50 con 0x7f582806c420 2023-10-30T12:57:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582814d5d0 con 0x7f582806c420 2023-10-30T12:57:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582814db90 con 0x7f582806c420 2023-10-30T12:57:18.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582806c190 con 0x7f582806c420 2023-10-30T12:57:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f582000acb0 con 0x7f582806c420 2023-10-30T12:57:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f581e7fc640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 0x7f580c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:18.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f5820083710 con 0x7f582806c420 2023-10-30T12:57:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f582d83a640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 0x7f580c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.726+0000 7f582d83a640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 0x7f580c068990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58240ae000 tx=0x7f58240a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:18.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.730+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5820050850 con 0x7f582806c420 2023-10-30T12:57:18.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.922+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f582804e170 con 0x7f582806c420 2023-10-30T12:57:18.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.922+0000 7f581e7fc640 1 -- 172.21.15.8:0/202666784 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f5820018090 con 0x7f582806c420 2023-10-30T12:57:18.929 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:18.930 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:13.613813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:resolve","state_seq":25,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:18.930 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 61 2023-10-30T12:57:18.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 msgr2=0x7f580c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 0x7f580c068990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58240ae000 tx=0x7f58240a7000 comp rx=0 tx=0).stop 2023-10-30T12:57:18.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 msgr2=0x7f582814e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:18.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f582000c7e0 tx=0x7f582000cca0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 shutdown_connections 2023-10-30T12:57:18.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f580c0664d0 0x7f580c068990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f582806bb40 0x7f582814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f582806c420 0x7f582814e390 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 --2- 172.21.15.8:0/202666784 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f582806b760 0x7f5828149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:18.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 >> 172.21.15.8:0/202666784 conn(0x7f58280cac70 msgr2=0x7f58281520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:18.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 shutdown_connections 2023-10-30T12:57:18.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:18.926+0000 7f582fa9e640 1 -- 172.21.15.8:0/202666784 wait complete. 2023-10-30T12:57:18.951 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:20.952 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 Processor -- start 2023-10-30T12:57:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 -- start start 2023-10-30T12:57:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0c2310 con 0x7f7d1c150830 2023-10-30T12:57:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c156710 con 0x7f7d1c0c3e40 2023-10-30T12:57:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d23a9e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c06bc10 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:21.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41334/0 (socket says 172.21.15.8:41334) 2023-10-30T12:57:21.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 -- 172.21.15.8:0/2860326771 learned_addr learned my addr 172.21.15.8:0/2860326771 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:21.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d2183a640 1 --1- 172.21.15.8:0/2860326771 >> v1:172.21.15.8:6789/0 conn(0x7f7d1c150830 0x7f7d1c06b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50038/0 (socket says 172.21.15.8:50038) 2023-10-30T12:57:21.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 -- 172.21.15.8:0/2860326771 >> v1:172.21.15.46:6789/0 conn(0x7f7d1c0c3e40 legacy=0x7f7d1c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:21.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 -- 172.21.15.8:0/2860326771 >> v1:172.21.15.8:6789/0 conn(0x7f7d1c150830 legacy=0x7f7d1c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d21039640 1 -- 172.21.15.8:0/2860326771 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0c4360 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.102+0000 7f7d20838640 1 -- 172.21.15.8:0/2860326771 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744735163 0 0) 0x7f7d1c0c2310 con 0x7f7d1c150830 2023-10-30T12:57:21.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d21039640 1 --2- 172.21.15.8:0/2860326771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00bb10 tx=0x7f7d0c0213c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76955fe6b76e064 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:21.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d20838640 1 -- 172.21.15.8:0/2860326771 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0b1600 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d20838640 1 -- 172.21.15.8:0/2860326771 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0ad050 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d20838640 1 -- 172.21.15.8:0/2860326771 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0b1980 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 -- 172.21.15.8:0/2860326771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 msgr2=0x7f7d1c0c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/2860326771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00bb10 tx=0x7f7d0c0213c0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 -- 172.21.15.8:0/2860326771 shutdown_connections 2023-10-30T12:57:21.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/2860326771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c0c3900 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 -- 172.21.15.8:0/2860326771 >> 172.21.15.8:0/2860326771 conn(0x7f7d1c0cac70 msgr2=0x7f7d1c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:21.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 -- 172.21.15.8:0/2860326771 shutdown_connections 2023-10-30T12:57:21.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.106+0000 7f7d23a9e640 1 -- 172.21.15.8:0/2860326771 wait complete. 2023-10-30T12:57:21.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 Processor -- start 2023-10-30T12:57:21.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- start start 2023-10-30T12:57:21.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7d1c0c3e40 0x7f7d1c147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c150830 0x7f7d1c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c158990 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0ca100 con 0x7f7d1c150830 2023-10-30T12:57:21.117 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c159fa0 con 0x7f7d1c0c3e40 2023-10-30T12:57:21.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:21.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41350/0 (socket says 172.21.15.8:41350) 2023-10-30T12:57:21.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 -- 172.21.15.8:0/729481825 learned_addr learned my addr 172.21.15.8:0/729481825 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:21.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d2183a640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7d1c0c3e40 0x7f7d1c147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d2203b640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c150830 0x7f7d1c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 -- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7d1c0c3e40 msgr2=0x7f7d1c147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7d1c0c3e40 0x7f7d1c147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 -- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c150830 msgr2=0x7f7d1c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c150830 0x7f7d1c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 -- 172.21.15.8:0/729481825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c00b770 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d21039640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b2c0 tx=0x7f7d0c01e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0bb670 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0bb7f0 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0bbba0 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c159530 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c0ca350 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.110+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c156f50 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.114+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d0c0b8040 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.114+0000 7f7d127fc640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 0x7f7d00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.114+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f7d0c1249f0 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.114+0000 7f7d2183a640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 0x7f7d00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.114+0000 7f7d2183a640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 0x7f7d00068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f70 tx=0x7f7d18007c30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:21.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.118+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d1c156f50 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.314+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d1c156f50 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.314+0000 7f7d127fc640 1 -- 172.21.15.8:0/729481825 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f7d1c156f50 con 0x7f7d1c0cb7d0 2023-10-30T12:57:21.321 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:21.321 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:18.958285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:21.321 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 62 2023-10-30T12:57:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 msgr2=0x7f7d00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 0x7f7d00068960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f70 tx=0x7f7d18007c30 comp rx=0 tx=0).stop 2023-10-30T12:57:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 msgr2=0x7f7d1c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b2c0 tx=0x7f7d0c01e160 comp rx=0 tx=0).stop 2023-10-30T12:57:21.323 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 shutdown_connections 2023-10-30T12:57:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7d000664a0 0x7f7d00068960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c150830 0x7f7d1c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d1c0cb7d0 0x7f7d1c1577c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 --2- 172.21.15.8:0/729481825 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7d1c0c3e40 0x7f7d1c147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:21.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 >> 172.21.15.8:0/729481825 conn(0x7f7d1c0cac70 msgr2=0x7f7d1c14d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:21.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 shutdown_connections 2023-10-30T12:57:21.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:21.318+0000 7f7d23a9e640 1 -- 172.21.15.8:0/729481825 wait complete. 2023-10-30T12:57:21.342 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 62, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-30T12:45:18.463017+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 62, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5266": { "addr": "172.21.15.46:6833/1405231055", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6832", "nonce": 1405231055, "type": "v2" }, { "addr": "172.21.15.46:6833", "nonce": 1405231055, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5266, "incarnation": 41, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 31 }, "gid_5394": { "addr": "172.21.15.8:6837/1849048959", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6834", "nonce": 1849048959, "type": "v2" }, { "addr": "172.21.15.8:6837", "nonce": 1849048959, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5394, "incarnation": 52, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5535": { "addr": "172.21.15.8:6839/4033248661", "addrs": { "addrvec": [ { "addr": "172.21.15.8:6836", "nonce": 4033248661, "type": "v2" }, { "addr": "172.21.15.8:6839", "nonce": 4033248661, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5535, "incarnation": 60, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-10-30T12:57:18.958285+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266, "mds_1": 5535, "mds_2": 5394 } } } ], "standbys": [ { "addr": "172.21.15.46:6836/4217977183", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6834", "nonce": 4217977183, "type": "v2" }, { "addr": "172.21.15.46:6836", "nonce": 4217977183, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4235, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.46:6837/3564454356", "addrs": { "addrvec": [ { "addr": "172.21.15.46:6835", "nonce": 3564454356, "type": "v2" }, { "addr": "172.21.15.46:6837", "nonce": 3564454356, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-30T12:57:23.343 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:23.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 Processor -- start 2023-10-30T12:57:23.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 -- start start 2023-10-30T12:57:23.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50e4070520 0x7f50e40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e40c2310 con 0x7f50e40cb7d0 2023-10-30T12:57:23.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e4156710 con 0x7f50e4070520 2023-10-30T12:57:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50ea809640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40c4360 con 0x7f50e4150830 2023-10-30T12:57:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e8da6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50e4070520 0x7f50e40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:60098/0 (socket says 172.21.15.8:60098) 2023-10-30T12:57:23.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 -- 172.21.15.8:0/3161233404 learned_addr learned my addr 172.21.15.8:0/3161233404 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:23.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 -- 172.21.15.8:0/3161233404 >> v2:172.21.15.46:3300/0 conn(0x7f50e4070520 msgr2=0x7f50e40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:23.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 --2- 172.21.15.8:0/3161233404 >> v2:172.21.15.46:3300/0 conn(0x7f50e4070520 0x7f50e40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 -- 172.21.15.8:0/3161233404 >> v1:172.21.15.46:6789/0 conn(0x7f50e40cb7d0 legacy=0x7f50e40c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.494+0000 7f50e3fff640 1 -- 172.21.15.8:0/3161233404 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e41569b0 con 0x7f50e4150830 2023-10-30T12:57:23.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50e3fff640 1 --2- 172.21.15.8:0/3161233404 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009870 tx=0x7f50d4020f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438b8c25ef0eff38 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50e2ffd640 1 -- 172.21.15.8:0/3161233404 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d40aac40 con 0x7f50e4150830 2023-10-30T12:57:23.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50e2ffd640 1 -- 172.21.15.8:0/3161233404 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d40aada0 con 0x7f50e4150830 2023-10-30T12:57:23.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50e2ffd640 1 -- 172.21.15.8:0/3161233404 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d40b2660 con 0x7f50e4150830 2023-10-30T12:57:23.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 -- 172.21.15.8:0/3161233404 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 msgr2=0x7f50e406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 --2- 172.21.15.8:0/3161233404 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009870 tx=0x7f50d4020f40 comp rx=0 tx=0).stop 2023-10-30T12:57:23.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 -- 172.21.15.8:0/3161233404 shutdown_connections 2023-10-30T12:57:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 --2- 172.21.15.8:0/3161233404 >> v2:172.21.15.46:3300/0 conn(0x7f50e4070520 0x7f50e40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 --2- 172.21.15.8:0/3161233404 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e4150830 0x7f50e406b500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 -- 172.21.15.8:0/3161233404 >> 172.21.15.8:0/3161233404 conn(0x7f50e40cac70 msgr2=0x7f50e40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:23.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 -- 172.21.15.8:0/3161233404 shutdown_connections 2023-10-30T12:57:23.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.498+0000 7f50ea809640 1 -- 172.21.15.8:0/3161233404 wait complete. 2023-10-30T12:57:23.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 Processor -- start 2023-10-30T12:57:23.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 -- start start 2023-10-30T12:57:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e4070520 0x7f50e414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e40cb7d0 0x7f50e41eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41eeaf0 con 0x7f50e4150830 2023-10-30T12:57:23.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4159c50 con 0x7f50e4070520 2023-10-30T12:57:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50ea809640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4159f30 con 0x7f50e40cb7d0 2023-10-30T12:57:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41364/0 (socket says 172.21.15.8:41364) 2023-10-30T12:57:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 -- 172.21.15.8:0/2640309048 learned_addr learned my addr 172.21.15.8:0/2640309048 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:23.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e37fe640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e40cb7d0 0x7f50e41eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e3fff640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e4070520 0x7f50e414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 -- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e40cb7d0 msgr2=0x7f50e41eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e40cb7d0 0x7f50e41eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 -- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e4070520 msgr2=0x7f50e414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e4070520 0x7f50e414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 -- 172.21.15.8:0/2640309048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40094d0 con 0x7f50e4150830 2023-10-30T12:57:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.502+0000 7f50e8da6640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00efc0 tx=0x7f50dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc014480 con 0x7f50e4150830 2023-10-30T12:57:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc00f040 con 0x7f50e4150830 2023-10-30T12:57:23.514 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc014820 con 0x7f50e4150830 2023-10-30T12:57:23.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e415a210 con 0x7f50e4150830 2023-10-30T12:57:23.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e4159950 con 0x7f50e4150830 2023-10-30T12:57:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e41eec60 con 0x7f50e4150830 2023-10-30T12:57:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f50dc014980 con 0x7f50e4150830 2023-10-30T12:57:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 0x7f50d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f50dc084e60 con 0x7f50e4150830 2023-10-30T12:57:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e3fff640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 0x7f50d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:23.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.506+0000 7f50e3fff640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 0x7f50d0068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50d40b4000 tx=0x7f50d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:23.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.510+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f50e41eec60 con 0x7f50e4150830 2023-10-30T12:57:23.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.702+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50e41eec30 con 0x7f50e4150830 2023-10-30T12:57:23.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.702+0000 7f50e0ff9640 1 -- 172.21.15.8:0/2640309048 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f50dc018090 con 0x7f50e4150830 2023-10-30T12:57:23.709 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:23.710 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:18.958285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 62 2023-10-30T12:57:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 msgr2=0x7f50d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 0x7f50d0068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50d40b4000 tx=0x7f50d4009040 comp rx=0 tx=0).stop 2023-10-30T12:57:23.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 msgr2=0x7f50e41ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00efc0 tx=0x7f50dc00c490 comp rx=0 tx=0).stop 2023-10-30T12:57:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 shutdown_connections 2023-10-30T12:57:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f50d00664a0 0x7f50d0068960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50e4150830 0x7f50e41ef480 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f50e40cb7d0 0x7f50e41eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 --2- 172.21.15.8:0/2640309048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e4070520 0x7f50e414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:23.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 >> 172.21.15.8:0/2640309048 conn(0x7f50e40cac70 msgr2=0x7f50e40c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:23.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 shutdown_connections 2023-10-30T12:57:23.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:23.706+0000 7f50ea809640 1 -- 172.21.15.8:0/2640309048 wait complete. 2023-10-30T12:57:23.730 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:25.732 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:25.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 Processor -- start 2023-10-30T12:57:25.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 -- start start 2023-10-30T12:57:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5880153c80 0x7f588015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880159f20 con 0x7f58800c42a0 2023-10-30T12:57:25.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588015d620 con 0x7f58800c4a10 2023-10-30T12:57:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f5885a46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588015dec0 con 0x7f5880153c80 2023-10-30T12:57:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5880153c80 0x7f588015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5880153c80 0x7f588015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41372/0 (socket says 172.21.15.8:41372) 2023-10-30T12:57:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 -- 172.21.15.8:0/2128803176 learned_addr learned my addr 172.21.15.8:0/2128803176 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:25.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587f7fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f58800c42a0 0x7f58800ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50042/0 (socket says 172.21.15.8:50042) 2023-10-30T12:57:25.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 -- 172.21.15.8:0/2128803176 >> v1:172.21.15.46:6789/0 conn(0x7f58800c4a10 legacy=0x7f588015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:25.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 -- 172.21.15.8:0/2128803176 >> v1:172.21.15.8:6789/0 conn(0x7f58800c42a0 legacy=0x7f58800ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:25.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.882+0000 7f587effd640 1 -- 172.21.15.8:0/2128803176 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588015d900 con 0x7f5880153c80 2023-10-30T12:57:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f587effd640 1 --2- 172.21.15.8:0/2128803176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588015cf90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5870009870 tx=0x7f587001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8fa04ed68e17f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:25.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f587e7fc640 1 -- 172.21.15.8:0/2128803176 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700a89f0 con 0x7f5880153c80 2023-10-30T12:57:25.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f587e7fc640 1 -- 172.21.15.8:0/2128803176 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700a8b50 con 0x7f5880153c80 2023-10-30T12:57:25.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f587e7fc640 1 -- 172.21.15.8:0/2128803176 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700b0440 con 0x7f5880153c80 2023-10-30T12:57:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 -- 172.21.15.8:0/2128803176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 msgr2=0x7f588015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 --2- 172.21.15.8:0/2128803176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588015cf90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5870009870 tx=0x7f587001ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 -- 172.21.15.8:0/2128803176 shutdown_connections 2023-10-30T12:57:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 --2- 172.21.15.8:0/2128803176 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588015cf90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:25.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 -- 172.21.15.8:0/2128803176 >> 172.21.15.8:0/2128803176 conn(0x7f58800715c0 msgr2=0x7f58800719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 -- 172.21.15.8:0/2128803176 shutdown_connections 2023-10-30T12:57:25.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.886+0000 7f5885a46640 1 -- 172.21.15.8:0/2128803176 wait complete. 2023-10-30T12:57:25.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 Processor -- start 2023-10-30T12:57:25.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- start start 2023-10-30T12:57:25.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58800c42a0 0x7f58801550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:25.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58800c4a10 0x7f5880149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:25.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:25.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880154c40 con 0x7f5880153c80 2023-10-30T12:57:25.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880156870 con 0x7f58800c4a10 2023-10-30T12:57:25.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880156b50 con 0x7f58800c42a0 2023-10-30T12:57:25.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:25.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41376/0 (socket says 172.21.15.8:41376) 2023-10-30T12:57:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 -- 172.21.15.8:0/767103053 learned_addr learned my addr 172.21.15.8:0/767103053 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587effd640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58800c42a0 0x7f58801550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5877fff640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58800c4a10 0x7f5880149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 -- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58800c42a0 msgr2=0x7f58801550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:25.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58800c42a0 0x7f58801550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 -- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58800c4a10 msgr2=0x7f5880149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58800c4a10 0x7f5880149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 -- 172.21.15.8:0/767103053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700094d0 con 0x7f5880153c80 2023-10-30T12:57:25.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587f7fe640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f587800c7a0 tx=0x7f587800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:25.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800a640 con 0x7f5880153c80 2023-10-30T12:57:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800a7a0 con 0x7f5880153c80 2023-10-30T12:57:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800ab10 con 0x7f5880153c80 2023-10-30T12:57:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5880154310 con 0x7f5880153c80 2023-10-30T12:57:25.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.890+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801548d0 con 0x7f5880153c80 2023-10-30T12:57:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588004e170 con 0x7f5880153c80 2023-10-30T12:57:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f587800ac70 con 0x7f5880153c80 2023-10-30T12:57:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f587cff9640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f5878083840 con 0x7f5880153c80 2023-10-30T12:57:25.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f587effd640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:25.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.894+0000 7f587effd640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58700b2000 tx=0x7f5870008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:25.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:25.898+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5878050ad0 con 0x7f5880153c80 2023-10-30T12:57:26.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.090+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58800cad40 con 0x7f5880153c80 2023-10-30T12:57:26.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.090+0000 7f587cff9640 1 -- 172.21.15.8:0/767103053 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f5878017090 con 0x7f5880153c80 2023-10-30T12:57:26.097 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:26.097 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:24.252972+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:26.098 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 63 2023-10-30T12:57:26.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 msgr2=0x7f5860068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:26.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58700b2000 tx=0x7f5870008040 comp rx=0 tx=0).stop 2023-10-30T12:57:26.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 msgr2=0x7f588014a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:26.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f587800c7a0 tx=0x7f587800cc60 comp rx=0 tx=0).stop 2023-10-30T12:57:26.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 shutdown_connections 2023-10-30T12:57:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f58600664a0 0x7f5860068960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58800c4a10 0x7f5880149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5880153c80 0x7f588014a0e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 --2- 172.21.15.8:0/767103053 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f58800c42a0 0x7f58801550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:26.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 >> 172.21.15.8:0/767103053 conn(0x7f58800715c0 msgr2=0x7f5880151750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:26.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 shutdown_connections 2023-10-30T12:57:26.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:26.094+0000 7f5885a46640 1 -- 172.21.15.8:0/767103053 wait complete. 2023-10-30T12:57:26.118 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:28.119 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:28.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 Processor -- start 2023-10-30T12:57:28.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 -- start start 2023-10-30T12:57:28.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d1815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d180c42a0 0x7f3d180ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d18159f20 con 0x7f3d18153c80 2023-10-30T12:57:28.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1815dec0 con 0x7f3d180c42a0 2023-10-30T12:57:28.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1f3c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d181599a0 con 0x7f3d180c4a10 2023-10-30T12:57:28.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1d161640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3d18153c80 0x7f3d1815a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:50054/0 (socket says 172.21.15.8:50054) 2023-10-30T12:57:28.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1d161640 1 -- 172.21.15.8:0/3727863385 learned_addr learned my addr 172.21.15.8:0/3727863385 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:28.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1c960640 1 --2- 172.21.15.8:0/3727863385 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d1815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d1d962640 1 --2- 172.21.15.8:0/3727863385 >> v2:172.21.15.46:3300/0 conn(0x7f3d180c42a0 0x7f3d180ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.270+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785405236 0 0) 0x7f3d18159f20 con 0x7f3d18153c80 2023-10-30T12:57:28.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d04003530 con 0x7f3d18153c80 2023-10-30T12:57:28.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d1c960640 1 -- 172.21.15.8:0/3727863385 >> v2:172.21.15.46:3300/0 conn(0x7f3d180c42a0 msgr2=0x7f3d180ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:28.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d1c960640 1 --2- 172.21.15.8:0/3727863385 >> v2:172.21.15.46:3300/0 conn(0x7f3d180c42a0 0x7f3d180ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d1c960640 1 -- 172.21.15.8:0/3727863385 >> v1:172.21.15.8:6789/0 conn(0x7f3d18153c80 legacy=0x7f3d1815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909447144 0 0) 0x7f3d04003530 con 0x7f3d18153c80 2023-10-30T12:57:28.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d1c960640 1 -- 172.21.15.8:0/3727863385 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1815d620 con 0x7f3d180c4a10 2023-10-30T12:57:28.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d1c960640 1 --2- 172.21.15.8:0/3727863385 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d1815cf90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097c0 tx=0x7f3d0801eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596d905f6a8dfc6f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:28.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14001680 con 0x7f3d180c4a10 2023-10-30T12:57:28.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d080a85d0 con 0x7f3d180c4a10 2023-10-30T12:57:28.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.274+0000 7f3d0ffff640 1 -- 172.21.15.8:0/3727863385 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d080a8960 con 0x7f3d180c4a10 2023-10-30T12:57:28.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/3727863385 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 msgr2=0x7f3d1815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/3727863385 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d1815cf90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097c0 tx=0x7f3d0801eb90 comp rx=0 tx=0).stop 2023-10-30T12:57:28.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/3727863385 shutdown_connections 2023-10-30T12:57:28.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/3727863385 >> v2:172.21.15.46:3300/0 conn(0x7f3d180c42a0 0x7f3d180ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/3727863385 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d1815cf90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/3727863385 >> 172.21.15.8:0/3727863385 conn(0x7f3d180715c0 msgr2=0x7f3d180719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:28.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/3727863385 shutdown_connections 2023-10-30T12:57:28.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/3727863385 wait complete. 2023-10-30T12:57:28.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 Processor -- start 2023-10-30T12:57:28.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- start start 2023-10-30T12:57:28.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d180c42a0 0x7f3d18155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d18154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1814f810 con 0x7f3d18153c80 2023-10-30T12:57:28.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1814e670 con 0x7f3d180c42a0 2023-10-30T12:57:28.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1f3c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1814e0b0 con 0x7f3d180c4a10 2023-10-30T12:57:28.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41388/0 (socket says 172.21.15.8:41388) 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 -- 172.21.15.8:0/2265011844 learned_addr learned my addr 172.21.15.8:0/2265011844 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d161640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d180c42a0 0x7f3d18155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1c960640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d18154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 -- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 msgr2=0x7f3d18154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d18154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 -- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d180c42a0 msgr2=0x7f3d18155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d180c42a0 0x7f3d18155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.278+0000 7f3d1d962640 1 -- 172.21.15.8:0/2265011844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08009420 con 0x7f3d18153c80 2023-10-30T12:57:28.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d1d962640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000efc0 tx=0x7f3d1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:28.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10014480 con 0x7f3d18153c80 2023-10-30T12:57:28.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1000f040 con 0x7f3d18153c80 2023-10-30T12:57:28.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10014820 con 0x7f3d18153c80 2023-10-30T12:57:28.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1814e390 con 0x7f3d18153c80 2023-10-30T12:57:28.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d18156c50 con 0x7f3d18153c80 2023-10-30T12:57:28.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d181567d0 con 0x7f3d18153c80 2023-10-30T12:57:28.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3d10014980 con 0x7f3d18153c80 2023-10-30T12:57:28.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.282+0000 7f3d0dffb640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 0x7f3cfc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:28.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.286+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f3d10084b10 con 0x7f3d18153c80 2023-10-30T12:57:28.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.286+0000 7f3d1d161640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 0x7f3cfc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:28.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.286+0000 7f3d1d161640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 0x7f3cfc068990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001ec0 tx=0x7f3d140019c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:28.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.290+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3d181567d0 con 0x7f3d18153c80 2023-10-30T12:57:28.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.478+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d18154240 con 0x7f3d18153c80 2023-10-30T12:57:28.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.478+0000 7f3d0dffb640 1 -- 172.21.15.8:0/2265011844 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f3d18154240 con 0x7f3d18153c80 2023-10-30T12:57:28.485 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:28.486 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:24.252972+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:28.486 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 63 2023-10-30T12:57:28.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 msgr2=0x7f3cfc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 0x7f3cfc068990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001ec0 tx=0x7f3d140019c0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 msgr2=0x7f3d1814f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:28.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000efc0 tx=0x7f3d1000c490 comp rx=0 tx=0).stop 2023-10-30T12:57:28.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 shutdown_connections 2023-10-30T12:57:28.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f3cfc0664d0 0x7f3cfc068990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d18153c80 0x7f3d1814f2d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f3d180c4a10 0x7f3d18154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 --2- 172.21.15.8:0/2265011844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d180c42a0 0x7f3d18155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:28.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 >> 172.21.15.8:0/2265011844 conn(0x7f3d180715c0 msgr2=0x7f3d18158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:28.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 shutdown_connections 2023-10-30T12:57:28.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:28.482+0000 7f3d1f3c5640 1 -- 172.21.15.8:0/2265011844 wait complete. 2023-10-30T12:57:28.506 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-30T12:57:30.508 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:30.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 Processor -- start 2023-10-30T12:57:30.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 -- start start 2023-10-30T12:57:30.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2af415d840 0x7f2af40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af40cd420 0x7f2af406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40c7b00 con 0x7f2af4153aa0 2023-10-30T12:57:30.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af4070a50 con 0x7f2af415d840 2023-10-30T12:57:30.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afc51d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40c2050 con 0x7f2af40cd420 2023-10-30T12:57:30.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2af415d840 0x7f2af40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2af415d840 0x7f2af40ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44900/0 (socket says 172.21.15.8:44900) 2023-10-30T12:57:30.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 -- 172.21.15.8:0/1619483157 learned_addr learned my addr 172.21.15.8:0/1619483157 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:30.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2afaaba640 1 --2- 172.21.15.8:0/1619483157 >> v2:172.21.15.46:3300/0 conn(0x7f2af40cd420 0x7f2af406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 -- 172.21.15.8:0/1619483157 >> v2:172.21.15.46:3300/0 conn(0x7f2af40cd420 msgr2=0x7f2af406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:30.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 --2- 172.21.15.8:0/1619483157 >> v2:172.21.15.46:3300/0 conn(0x7f2af40cd420 0x7f2af406b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 -- 172.21.15.8:0/1619483157 >> v1:172.21.15.46:6789/0 conn(0x7f2af4153aa0 legacy=0x7f2af40701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.658+0000 7f2af9ab8640 1 -- 172.21.15.8:0/1619483157 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40c2330 con 0x7f2af415d840 2023-10-30T12:57:30.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2af9ab8640 1 --2- 172.21.15.8:0/1619483157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af40ccee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2af40cdf10 tx=0x7f2ae401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24818cedd890ede5 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:30.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2af92b7640 1 -- 172.21.15.8:0/1619483157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40a89f0 con 0x7f2af415d840 2023-10-30T12:57:30.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2af92b7640 1 -- 172.21.15.8:0/1619483157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40a8b50 con 0x7f2af415d840 2023-10-30T12:57:30.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2af92b7640 1 -- 172.21.15.8:0/1619483157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b0440 con 0x7f2af415d840 2023-10-30T12:57:30.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 -- 172.21.15.8:0/1619483157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 msgr2=0x7f2af40ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 --2- 172.21.15.8:0/1619483157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af40ccee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2af40cdf10 tx=0x7f2ae401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 -- 172.21.15.8:0/1619483157 shutdown_connections 2023-10-30T12:57:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 --2- 172.21.15.8:0/1619483157 >> v2:172.21.15.46:3300/0 conn(0x7f2af40cd420 0x7f2af406b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 --2- 172.21.15.8:0/1619483157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af40ccee0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 -- 172.21.15.8:0/1619483157 >> 172.21.15.8:0/1619483157 conn(0x7f2af4071fb0 msgr2=0x7f2af40723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:30.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 -- 172.21.15.8:0/1619483157 shutdown_connections 2023-10-30T12:57:30.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.662+0000 7f2afc51d640 1 -- 172.21.15.8:0/1619483157 wait complete. 2023-10-30T12:57:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 Processor -- start 2023-10-30T12:57:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- start start 2023-10-30T12:57:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2af40cd420 0x7f2af4154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4153aa0 0x7f2af41f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4157c60 con 0x7f2af415d840 2023-10-30T12:57:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40ccad0 con 0x7f2af4153aa0 2023-10-30T12:57:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40cd2a0 con 0x7f2af40cd420 2023-10-30T12:57:30.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44914/0 (socket says 172.21.15.8:44914) 2023-10-30T12:57:30.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 -- 172.21.15.8:0/2315763398 learned_addr learned my addr 172.21.15.8:0/2315763398 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:30.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2af9ab8640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4153aa0 0x7f2af41f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afa2b9640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2af40cd420 0x7f2af4154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 -- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2af40cd420 msgr2=0x7f2af4154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2af40cd420 0x7f2af4154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 -- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4153aa0 msgr2=0x7f2af41f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4153aa0 0x7f2af41f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 -- 172.21.15.8:0/2315763398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40094d0 con 0x7f2af415d840 2023-10-30T12:57:30.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afaaba640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007e40 tx=0x7f2aec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:30.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec014620 con 0x7f2af415d840 2023-10-30T12:57:30.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec00ee90 con 0x7f2af415d840 2023-10-30T12:57:30.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0149e0 con 0x7f2af415d840 2023-10-30T12:57:30.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af41f30e0 con 0x7f2af415d840 2023-10-30T12:57:30.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.666+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af40caf50 con 0x7f2af415d840 2023-10-30T12:57:30.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af40cadb0 con 0x7f2af415d840 2023-10-30T12:57:30.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2aec014b40 con 0x7f2af415d840 2023-10-30T12:57:30.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2aeaffd640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 0x7f2ad8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:30.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f2aec084b40 con 0x7f2af415d840 2023-10-30T12:57:30.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2afa2b9640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 0x7f2ad8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:30.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.670+0000 7f2afa2b9640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 0x7f2ad8068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001650 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:30.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.674+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2af40cadb0 con 0x7f2af415d840 2023-10-30T12:57:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af40cadb0 con 0x7f2af415d840 2023-10-30T12:57:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2aeaffd640 1 -- 172.21.15.8:0/2315763398 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f2af40cadb0 con 0x7f2af415d840 2023-10-30T12:57:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:30.869 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:29.814420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:active","state_seq":31,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:30.869 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 64 2023-10-30T12:57:30.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 msgr2=0x7f2ad8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 0x7f2ad8068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001650 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 msgr2=0x7f2af4149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.862+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007e40 tx=0x7f2aec00e4b0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 shutdown_connections 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f2ad80664a0 0x7f2ad8068960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4153aa0 0x7f2af41f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2af415d840 0x7f2af4149c30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 --2- 172.21.15.8:0/2315763398 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f2af40cd420 0x7f2af4154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:30.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 >> 172.21.15.8:0/2315763398 conn(0x7f2af4071fb0 msgr2=0x7f2af406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:30.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 shutdown_connections 2023-10-30T12:57:30.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:30.866+0000 7f2afc51d640 1 -- 172.21.15.8:0/2315763398 wait complete. 2023-10-30T12:57:30.890 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=5079 2023-10-30T12:57:30.890 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.e 2023-10-30T12:57:32.891 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-30T12:57:32.891 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-30T12:57:32.892 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-30T12:57:32.896 INFO:tasks.ceph.mds.e:Started 2023-10-30T12:57:32.896 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:32.994 INFO:tasks.ceph.mds.e.smithi008.stdout:starting mds.e at 2023-10-30T12:57:33.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 Processor -- start 2023-10-30T12:57:33.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 -- start start 2023-10-30T12:57:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 0x7f6dcc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6dcc15ac70 0x7f6dcc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc15dcc0 con 0x7f6dcc06aeb0 2023-10-30T12:57:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc159800 con 0x7f6dcc15ac70 2023-10-30T12:57:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd3288640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc159d80 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd2a87640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f6dcc06aeb0 0x7f6dcc069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42712/0 (socket says 172.21.15.8:42712) 2023-10-30T12:57:33.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1a85640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6dcc15ac70 0x7f6dcc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd2a87640 1 -- 172.21.15.8:0/2067635322 learned_addr learned my addr 172.21.15.8:0/2067635322 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd2286640 1 --2- 172.21.15.8:0/2067635322 >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 0x7f6dcc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1a85640 1 -- 172.21.15.8:0/2067635322 >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 msgr2=0x7f6dcc15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:33.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1a85640 1 --2- 172.21.15.8:0/2067635322 >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 0x7f6dcc15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1a85640 1 -- 172.21.15.8:0/2067635322 >> v1:172.21.15.8:6789/0 conn(0x7f6dcc06aeb0 legacy=0x7f6dcc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1a85640 1 -- 172.21.15.8:0/2067635322 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc15a750 con 0x7f6dcc15ac70 2023-10-30T12:57:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd2286640 1 --2- 172.21.15.8:0/2067635322 >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 0x7f6dcc15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:33.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.046+0000 7f6dd1284640 1 -- 172.21.15.8:0/2067635322 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694510066 0 0) 0x7f6dc40014a0 con 0x7f6dcc06aeb0 2023-10-30T12:57:33.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd1a85640 1 --2- 172.21.15.8:0/2067635322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc15a210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc06aa40 tx=0x7f6dbc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5df14eefa483cb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:33.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd1284640 1 -- 172.21.15.8:0/2067635322 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0a8950 con 0x7f6dcc15ac70 2023-10-30T12:57:33.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd1284640 1 -- 172.21.15.8:0/2067635322 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0a8ab0 con 0x7f6dcc15ac70 2023-10-30T12:57:33.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd1284640 1 -- 172.21.15.8:0/2067635322 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0a8e40 con 0x7f6dcc15ac70 2023-10-30T12:57:33.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 -- 172.21.15.8:0/2067635322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc15ac70 msgr2=0x7f6dcc15a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 --2- 172.21.15.8:0/2067635322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc15a210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc06aa40 tx=0x7f6dbc01ec50 comp rx=0 tx=0).stop 2023-10-30T12:57:33.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 -- 172.21.15.8:0/2067635322 shutdown_connections 2023-10-30T12:57:33.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 --2- 172.21.15.8:0/2067635322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc15a210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 --2- 172.21.15.8:0/2067635322 >> v2:172.21.15.46:3300/0 conn(0x7f6dcc0c8eb0 0x7f6dcc15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 -- 172.21.15.8:0/2067635322 >> 172.21.15.8:0/2067635322 conn(0x7f6dcc0cef00 msgr2=0x7f6dcc0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:33.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 -- 172.21.15.8:0/2067635322 shutdown_connections 2023-10-30T12:57:33.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.050+0000 7f6dd3288640 1 -- 172.21.15.8:0/2067635322 wait complete. 2023-10-30T12:57:33.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 Processor -- start 2023-10-30T12:57:33.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- start start 2023-10-30T12:57:33.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6dcc06aeb0 0x7f6dcc1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc155e00 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1f1db0 con 0x7f6dcc15ac70 2023-10-30T12:57:33.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1559c0 con 0x7f6dcc06aeb0 2023-10-30T12:57:33.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44944/0 (socket says 172.21.15.8:44944) 2023-10-30T12:57:33.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 -- 172.21.15.8:0/1801249338 learned_addr learned my addr 172.21.15.8:0/1801249338 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:33.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd2a87640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd2286640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6dcc06aeb0 0x7f6dcc1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 -- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6dcc06aeb0 msgr2=0x7f6dcc1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6dcc06aeb0 0x7f6dcc1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 -- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15ac70 msgr2=0x7f6dcc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 -- 172.21.15.8:0/1801249338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009420 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd1a85640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0b3040 tx=0x7f6dbc01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:33.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0b9630 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0b97b0 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc14e4f0 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0b9b60 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc14e790 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.054+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc04e170 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.058+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6dbc0b7040 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.058+0000 7f6dc2ffd640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 0x7f6db0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:33.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.058+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f6dbc1221e0 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.058+0000 7f6dd2286640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 0x7f6db0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:33.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.058+0000 7f6dd2286640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 0x7f6db0068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c900 tx=0x7f6dc8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:33.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.062+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6dbc0ef320 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.250+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dcc0ca960 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.250+0000 7f6dc2ffd640 1 -- 172.21.15.8:0/1801249338 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f6dbc0f2940 con 0x7f6dcc0c8eb0 2023-10-30T12:57:33.257 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:33.257 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:29.814420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:active","state_seq":31,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:33.257 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 64 2023-10-30T12:57:33.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 msgr2=0x7f6db0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 0x7f6db0068960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c900 tx=0x7f6dc8008040 comp rx=0 tx=0).stop 2023-10-30T12:57:33.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 msgr2=0x7f6dcc1f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:33.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0b3040 tx=0x7f6dbc01b9f0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 shutdown_connections 2023-10-30T12:57:33.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6db00664a0 0x7f6db0068960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc15ac70 0x7f6dcc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6dcc0c8eb0 0x7f6dcc1f2520 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 --2- 172.21.15.8:0/1801249338 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6dcc06aeb0 0x7f6dcc1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:33.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 >> 172.21.15.8:0/1801249338 conn(0x7f6dcc0cef00 msgr2=0x7f6dcc0c69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:33.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 shutdown_connections 2023-10-30T12:57:33.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:33.254+0000 7f6dd3288640 1 -- 172.21.15.8:0/1801249338 wait complete. 2023-10-30T12:57:33.280 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-30T12:57:35.280 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:57:35.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 Processor -- start 2023-10-30T12:57:35.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 -- start start 2023-10-30T12:57:35.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe8dc1537f0 0x7fe8dc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc15a2e0 0x7fe8dc0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0c2aa0 con 0x7fe8dc0cd4a0 2023-10-30T12:57:35.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c3ab0 con 0x7fe8dc1537f0 2023-10-30T12:57:35.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e37ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0706e0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe8dc1537f0 0x7fe8dc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe8dc1537f0 0x7fe8dc0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44946/0 (socket says 172.21.15.8:44946) 2023-10-30T12:57:35.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 -- 172.21.15.8:0/2452459368 learned_addr learned my addr 172.21.15.8:0/2452459368 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:35.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e0d46640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc15a2e0 0x7fe8dc0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 -- 172.21.15.8:0/2452459368 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc15a2e0 msgr2=0x7fe8dc0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:35.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc15a2e0 0x7fe8dc0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 -- 172.21.15.8:0/2452459368 >> v1:172.21.15.46:6789/0 conn(0x7fe8dc0cd4a0 legacy=0x7fe8dc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.446+0000 7fe8e1547640 1 -- 172.21.15.8:0/2452459368 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0708e0 con 0x7fe8dc1537f0 2023-10-30T12:57:35.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e1547640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc1537f0 0x7fe8dc0701a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009870 tx=0x7fe8d801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0477b46e8ca613 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:35.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8d3fff640 1 -- 172.21.15.8:0/2452459368 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80a8a30 con 0x7fe8dc1537f0 2023-10-30T12:57:35.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8d3fff640 1 -- 172.21.15.8:0/2452459368 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d80a8b90 con 0x7fe8dc1537f0 2023-10-30T12:57:35.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8d3fff640 1 -- 172.21.15.8:0/2452459368 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80b0440 con 0x7fe8dc1537f0 2023-10-30T12:57:35.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2452459368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc1537f0 msgr2=0x7fe8dc0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc1537f0 0x7fe8dc0701a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009870 tx=0x7fe8d801ed30 comp rx=0 tx=0).stop 2023-10-30T12:57:35.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2452459368 shutdown_connections 2023-10-30T12:57:35.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc15a2e0 0x7fe8dc0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2452459368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc1537f0 0x7fe8dc0701a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2452459368 >> 172.21.15.8:0/2452459368 conn(0x7fe8dc071fb0 msgr2=0x7fe8dc0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:35.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2452459368 shutdown_connections 2023-10-30T12:57:35.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2452459368 wait complete. 2023-10-30T12:57:35.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.450+0000 7fe8e37ab640 1 Processor -- start 2023-10-30T12:57:35.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- start start 2023-10-30T12:57:35.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8dc0cd4a0 0x7fe8dc158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc1537f0 0x7fe8dc14c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c7120 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc158730 con 0x7fe8dc0cd4a0 2023-10-30T12:57:35.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc159a30 con 0x7fe8dc1537f0 2023-10-30T12:57:35.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44948/0 (socket says 172.21.15.8:44948) 2023-10-30T12:57:35.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e0d46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc1537f0 0x7fe8dc14c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 -- 172.21.15.8:0/2897799523 learned_addr learned my addr 172.21.15.8:0/2897799523 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:35.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1547640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8dc0cd4a0 0x7fe8dc158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 -- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc1537f0 msgr2=0x7fe8dc14c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc1537f0 0x7fe8dc14c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 -- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8dc0cd4a0 msgr2=0x7fe8dc158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8dc0cd4a0 0x7fe8dc158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 -- 172.21.15.8:0/2897799523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80094d0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e1d48640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c730 tx=0x7fe8d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:35.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4003b80 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4003ce0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400aa30 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc159100 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc159f70 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.454+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc1588a0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.458+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe8d400ab90 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.458+0000 7fe8d1ffb640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 0x7fe8c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:35.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.458+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fe8d4083ec0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.458+0000 7fe8e1547640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 0x7fe8c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:35.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.458+0000 7fe8e1547640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 0x7fe8c0068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b2000 tx=0x7fe8d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:35.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.462+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe8d40510b0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.650+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8dc04e170 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8d1ffb640 1 -- 172.21.15.8:0/2897799523 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6520 (secure 0 0 0) 0x7fe8d40546d0 con 0x7fe8dc15a2e0 2023-10-30T12:57:35.658 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:35.659 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5688,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/1050115493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":1050115493},{"type":"v1","addr":"172.21.15.8:6838","nonce":1050115493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:29.814420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:active","state_seq":31,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:57:35.659 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 65 2023-10-30T12:57:35.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 msgr2=0x7fe8c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 0x7fe8c0068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b2000 tx=0x7fe8d8009040 comp rx=0 tx=0).stop 2023-10-30T12:57:35.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 msgr2=0x7fe8dc14c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:35.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c730 tx=0x7fe8d400cbf0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 shutdown_connections 2023-10-30T12:57:35.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe8c00664a0 0x7fe8c0068960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8dc15a2e0 0x7fe8dc14c610 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe8dc1537f0 0x7fe8dc14c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 --2- 172.21.15.8:0/2897799523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8dc0cd4a0 0x7fe8dc158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:35.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 >> 172.21.15.8:0/2897799523 conn(0x7fe8dc071fb0 msgr2=0x7fe8dc06c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:35.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 shutdown_connections 2023-10-30T12:57:35.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:35.654+0000 7fe8e37ab640 1 -- 172.21.15.8:0/2897799523 wait complete. 2023-10-30T12:57:35.679 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-30T12:57:35.679 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 46 secs before thrashing 2023-10-30T12:57:36.327 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 0% (57/12071) Updating files: 1% (121/12071) Updating files: 2% (242/12071) Updating files: 3% (363/12071) Updating files: 3% (424/12071) Updating files: 3% (460/12071) Updating files: 4% (483/12071) Updating files: 5% (604/12071) Updating files: 5% (695/12071) Updating files: 6% (725/12071) Updating files: 6% (733/12071) Updating files: 7% (845/12071) Updating files: 7% (850/12071) Updating files: 7% (919/12071) Updating files: 8% (966/12071) Updating files: 8% (1016/12071) Updating files: 9% (1087/12071) Updating files: 10% (1208/12071) Updating files: 11% (1328/12071) Updating files: 11% (1370/12071) Updating files: 11% (1415/12071) Updating files: 12% (1449/12071) Updating files: 12% (1556/12071) Updating files: 13% (1570/12071) Updating files: 14% (1690/12071) Updating files: 14% (1767/12071) Updating files: 14% (1803/12071) Updating files: 15% (1811/12071) Updating files: 15% (1864/12071) Updating files: 15% (1925/12071) Updating files: 16% (1932/12071) Updating files: 17% (2053/12071) Updating files: 17% (2056/12071) Updating files: 17% (2090/12071) Updating files: 17% (2169/12071) Updating files: 18% (2173/12071) Updating files: 18% (2268/12071) Updating files: 18% (2284/12071) Updating files: 19% (2294/12071) Updating files: 20% (2415/12071) Updating files: 20% (2513/12071) Updating files: 21% (2535/12071) Updating files: 21% (2636/12071) Updating files: 22% (2656/12071) Updating files: 22% (2722/12071) Updating files: 22% (2741/12071) Updating files: 22% (2770/12071) Updating files: 23% (2777/12071) Updating files: 24% (2898/12071) Updating files: 24% (2960/12071) Updating files: 24% (2995/12071) Updating files: 25% (3018/12071) Updating files: 26% (3139/12071) Updating files: 26% (3204/12071) Updating files: 27% (3260/12071) Updating files: 27% (3368/12071) Updating files: 27% (3374/12071) Updating files: 28% (3380/12071) Updating files: 29% (3501/12071) Updating files: 29% (3607/12071) Updating files: 30% (3622/12071) Updating files: 30% (3718/12071) Updating files: 31% (3743/12071) Updating files: 31% (3800/12071) Updating files: 32% (3863/12071) Updating files: 33% (3984/12071) Updating files: 33% (4067/12071) Updating files: 33% (4092/12071) Updating files: 34% (4105/12071) Updating files: 34% (4216/12071) Updating files: 35% (4225/12071) Updating files: 35% (4267/12071) Updating files: 36% (4346/12071) Updating files: 36% (4389/12071) Updating files: 36% (4420/12071) Updating files: 37% (4467/12071) Updating files: 38% (4587/12071) Updating files: 38% (4630/12071) Updating files: 38% (4658/12071) Updating files: 38% (4678/12071) Updating files: 39% (4708/12071) Updating files: 39% (4811/12071) Updating files: 40% (4829/12071) Updating files: 40% (4910/12071) Updating files: 41% (4950/12071) Updating files: 41% (4986/12071) Updating files: 41% (5022/12071) Updating files: 41% (5024/12071) Updating files: 42% (5070/12071) Updating files: 42% (5117/12071) Updating files: 42% (5159/12071) Updating files: 43% (5191/12071) Updating files: 43% (5198/12071) Updating files: 43% (5277/12071) Updating files: 44% (5312/12071) Updating files: 45% (5432/12071) Updating files: 45% (5449/12071) Updating files: 46% (5553/12071) Updating files: 47% (5674/12071) Updating files: 47% (5678/12071) Updating files: 48% (5795/12071) Updating files: 48% (5832/12071) Updating files: 49% (5915/12071) Updating files: 49% (5975/12071) Updating files: 50% (6036/12071) Updating files: 50% (6037/12071) Updating files: 51% (6157/12071) Updating files: 51% (6189/12071) Updating files: 52% (6277/12071) Updating files: 52% (6371/12071) Updating files: 53% (6398/12071) Updating files: 54% (6519/12071) Updating files: 54% (6558/12071) Updating files: 54% (6577/12071) Updating files: 55% (6640/12071) Updating files: 55% (6653/12071) Updating files: 55% (6721/12071) Updating files: 56% (6760/12071) Updating files: 57% (6881/12071) Updating files: 57% (6963/12071) Updating files: 58% (7002/12071) Updating files: 58% (7095/12071) Updating files: 58% (7096/12071) Updating files: 59% (7122/12071) Updating files: 59% (7198/12071) Updating files: 60% (7243/12071) Updating files: 61% (7364/12071) Updating files: 61% (7403/12071) Updating files: 62% (7485/12071) Updating files: 62% (7553/12071) Updating files: 63% (7605/12071) Updating files: 64% (7726/12071) Updating files: 64% (7787/12071) Updating files: 64% (7813/12071) Updating files: 65% (7847/12071) Updating files: 66% (7967/12071) Updating files: 66% (8020/12071) Updating files: 66% (8071/12071) Updating files: 67% (8088/12071) Updating files: 68% (8209/12071) Updating files: 68% (8273/12071) Updating files: 69% (8329/12071) Updating files: 69% (8353/12071) Updating files: 70% (8450/12071) Updating files: 70% (8455/12071) Updating files: 70% (8463/12071) Updating files: 70% (8525/12071) Updating files: 71% (8571/12071) Updating files: 72% (8692/12071) Updating files: 72% (8748/12071) Updating files: 72% (8782/12071) Updating files: 73% (8812/12071) Updating files: 74% (8933/12071) Updating files: 75% (9054/12071) Updating files: 75% (9061/12071) Updating files: 75% (9136/12071) Updating files: 76% (9174/12071) Updating files: 77% (9295/12071) Updating files: 77% (9314/12071) Updating files: 78% (9416/12071) Updating files: 78% (9535/12071) Updating files: 79% (9537/12071) Updating files: 79% (9614/12071) Updating files: 80% (9657/12071) Updating files: 80% (9703/12071) Updating files: 80% (9734/12071) Updating files: 80% (9741/12071) Updating files: 80% (9753/12071) Updating files: 81% (9778/12071) Updating files: 82% (9899/12071) Updating files: 83% (10019/12071) Updating files: 84% (10140/12071) Updating files: 85% (10261/12071) Updating files: 85% (10273/12071) Updating files: 85% (10347/12071) Updating files: 86% (10382/12071) Updating files: 86% (10413/12071) Updating files: 86% (10436/12071) Updating files: 87% (10502/12071) Updating files: 87% (10620/12071) Updating files: 88% (10623/12071) Updating files: 88% (10731/12071) Updating files: 89% (10744/12071) Updating files: 90% (10864/12071) Updating files: 91% (10985/12071) Updating files: 91% (10991/12071) Updating files: 92% (11106/12071) Updating files: 92% (11137/12071) Updating files: 92% (11203/12071) Updating files: 93% (11227/12071) Updating files: 93% (11308/12071) Updating files: 93% (11321/12071) Updating files: 94% (11347/12071) Updating files: 94% (11351/12071) Updating files: 95% (11468/12071) Updating files: 95% (11482/12071) Updating files: 96% (11589/12071) Updating files: 96% (11659/12071) Updating files: 97% (11709/12071) Updating files: 97% (11738/12071) Updating files: 98% (11830/12071) Updating files: 98% (11832/12071) Updating files: 99% (11951/12071) Updating files: 99% (11963/12071) Updating files: 99% (12011/12071) Updating files: 99% (12038/12071) Updating files: 100% (12071/12071) Updating files: 100% (12071/12071), done. 2023-10-30T12:57:36.467 DEBUG:teuthology.orchestra.run:got remote process result: 128 2023-10-30T12:57:36.468 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2023-10-30T12:57:36.469 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-30T12:57:37.350 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi008 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-30T12:57:37.421 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e9109f81da8f44398e6a4e79673d840d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_00c1c13cea21a686b046584596fbeac5315a7d71/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi008 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-30T12:57:37.424 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2023-10-30T12:57:37.435 INFO:tasks.mds_thrash:joining mds_thrasher 2023-10-30T12:57:37.438 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 0 2023-10-30T12:57:37.438 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2023-10-30T12:57:37.438 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 6 2023-10-30T12:57:37.439 INFO:tasks.mds_thrash:done joining 2023-10-30T12:57:37.439 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-30T12:57:37.447 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-30T12:57:37.448 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:57:37.448 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:57:37.469 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-30T12:57:37.470 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-30T12:57:37.518 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:57:37.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.698+0000 7f67c889e640 1 Processor -- start 2023-10-30T12:57:37.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.698+0000 7f67c889e640 1 -- start start 2023-10-30T12:57:37.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.698+0000 7f67c889e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f67c4151840 0x7f67c40c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:37.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c889e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40c5030 con 0x7f67c4151c20 2023-10-30T12:57:37.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c889e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c4157830 con 0x7f67c40cb8b0 2023-10-30T12:57:37.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c889e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c4157b10 con 0x7f67c4151840 2023-10-30T12:57:37.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c259c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f67c4151840 0x7f67c40c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:37.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c1d9b640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f67c4151c20 0x7f67c4072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42718/0 (socket says 172.21.15.8:42718) 2023-10-30T12:57:37.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c1d9b640 1 -- 172.21.15.8:0/4054446742 learned_addr learned my addr 172.21.15.8:0/4054446742 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:37.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c259c640 1 -- 172.21.15.8:0/4054446742 >> v1:172.21.15.46:6789/0 conn(0x7f67c40cb8b0 legacy=0x7f67c406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:37.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c259c640 1 -- 172.21.15.8:0/4054446742 >> v1:172.21.15.8:6789/0 conn(0x7f67c4151c20 legacy=0x7f67c4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c259c640 1 -- 172.21.15.8:0/4054446742 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4157df0 con 0x7f67c4151840 2023-10-30T12:57:37.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c159a640 1 -- 172.21.15.8:0/4054446742 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756658835 0 0) 0x7f67c40c5030 con 0x7f67c4151c20 2023-10-30T12:57:37.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c259c640 1 --2- 172.21.15.8:0/4054446742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c40c49a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f67b40097c0 tx=0x7f67b401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4cf16063c5eae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:37.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c159a640 1 -- 172.21.15.8:0/4054446742 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a8930 con 0x7f67c4151840 2023-10-30T12:57:37.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c159a640 1 -- 172.21.15.8:0/4054446742 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b40a8a90 con 0x7f67c4151840 2023-10-30T12:57:37.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.702+0000 7f67c159a640 1 -- 172.21.15.8:0/4054446742 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a8e20 con 0x7f67c4151840 2023-10-30T12:57:37.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- 172.21.15.8:0/4054446742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 msgr2=0x7f67c40c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 --2- 172.21.15.8:0/4054446742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c40c49a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f67b40097c0 tx=0x7f67b401ec30 comp rx=0 tx=0).stop 2023-10-30T12:57:37.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- 172.21.15.8:0/4054446742 shutdown_connections 2023-10-30T12:57:37.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 --2- 172.21.15.8:0/4054446742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c40c49a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- 172.21.15.8:0/4054446742 >> 172.21.15.8:0/4054446742 conn(0x7f67c40cac70 msgr2=0x7f67c40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:37.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- 172.21.15.8:0/4054446742 shutdown_connections 2023-10-30T12:57:37.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- 172.21.15.8:0/4054446742 wait complete. 2023-10-30T12:57:37.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 Processor -- start 2023-10-30T12:57:37.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- start start 2023-10-30T12:57:37.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 0x7f67c414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:37.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:37.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67c4151c20 0x7f67c41f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:37.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41eeec0 con 0x7f67c4151840 2023-10-30T12:57:37.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41e1b90 con 0x7f67c40cb8b0 2023-10-30T12:57:37.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c889e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40ca950 con 0x7f67c4151c20 2023-10-30T12:57:37.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c1d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:37.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c1d9b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44968/0 (socket says 172.21.15.8:44968) 2023-10-30T12:57:37.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c1d9b640 1 -- 172.21.15.8:0/3269061749 learned_addr learned my addr 172.21.15.8:0/3269061749 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:37.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c259c640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 0x7f67c414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:37.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.706+0000 7f67c2d9d640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67c4151c20 0x7f67c41f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:37.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 -- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67c4151c20 msgr2=0x7f67c41f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67c4151c20 0x7f67c41f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 -- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 msgr2=0x7f67c414d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 0x7f67c414d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 -- 172.21.15.8:0/3269061749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4009420 con 0x7f67c4151840 2023-10-30T12:57:37.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c259c640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 0x7f67c414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:37.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c1d9b640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c4b0 tx=0x7f67b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:37.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80074e0 con 0x7f67c4151840 2023-10-30T12:57:37.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8007660 con 0x7f67c4151840 2023-10-30T12:57:37.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004a30 con 0x7f67c4151840 2023-10-30T12:57:37.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c40ca3d0 con 0x7f67c4151840 2023-10-30T12:57:37.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c40ca670 con 0x7f67c4151840 2023-10-30T12:57:37.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c40c66a0 con 0x7f67c4151840 2023-10-30T12:57:37.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67b8004b90 con 0x7f67c4151840 2023-10-30T12:57:37.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 0x7f67a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:37.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f67b80832e0 con 0x7f67c4151840 2023-10-30T12:57:37.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.710+0000 7f67c259c640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 0x7f67a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:37.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.714+0000 7f67c259c640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 0x7f67a4068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67b40b2000 tx=0x7f67b40aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:37.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.718+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67b8053eb0 con 0x7f67c4151840 2023-10-30T12:57:37.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.878+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c404e170 con 0x7f67c4151840 2023-10-30T12:57:37.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.878+0000 7f67b2ffd640 1 -- 172.21.15.8:0/3269061749 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17471 (secure 0 0 0) 0x7f67b8053a40 con 0x7f67c4151840 2023-10-30T12:57:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:37.886 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":53,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:57:07.233484+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6838/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.8:6839/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.8:6837/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.46:6833/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.46:6832/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6838/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6839/710307090":"2023-10-31T12:47:22.043669+0000","172.21.15.8:6834/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/710307090":"2023-10-31T12:47:22.043669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:57:37.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 msgr2=0x7f67a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 0x7f67a4068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67b40b2000 tx=0x7f67b40aa070 comp rx=0 tx=0).stop 2023-10-30T12:57:37.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 msgr2=0x7f67c41f2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:37.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c4b0 tx=0x7f67b800c970 comp rx=0 tx=0).stop 2023-10-30T12:57:37.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 shutdown_connections 2023-10-30T12:57:37.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f67a40664a0 0x7f67a4068960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f67c4151c20 0x7f67c41f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67c4151840 0x7f67c41f2a40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 --2- 172.21.15.8:0/3269061749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c40cb8b0 0x7f67c414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:37.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 >> 172.21.15.8:0/3269061749 conn(0x7f67c40cac70 msgr2=0x7f67c406b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:37.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 shutdown_connections 2023-10-30T12:57:37.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:37.882+0000 7f67c889e640 1 -- 172.21.15.8:0/3269061749 wait complete. 2023-10-30T12:57:37.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T12:45:05.671325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-30T12:45:16.140516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-30T12:45:17.462529+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T12:57:37.909 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T12:57:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 Processor -- start 2023-10-30T12:57:38.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 -- start start 2023-10-30T12:57:38.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95840c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958415d0a0 con 0x7f9584153300 2023-10-30T12:57:38.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958415d380 con 0x7f9584071cb0 2023-10-30T12:57:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95898dc640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840cca10 con 0x7f95840718d0 2023-10-30T12:57:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95890db640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f9584153300 0x7f95840cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42734/0 (socket says 172.21.15.8:42734) 2023-10-30T12:57:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95890db640 1 -- 172.21.15.8:0/3571617418 learned_addr learned my addr 172.21.15.8:0/3571617418 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:38.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95888da640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95840c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617611419 0 0) 0x7f958415d0a0 con 0x7f9584153300 2023-10-30T12:57:38.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9574003530 con 0x7f9584153300 2023-10-30T12:57:38.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600597029 0 0) 0x7f9574003530 con 0x7f9584153300 2023-10-30T12:57:38.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 msgr2=0x7f95840c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95840c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 >> v1:172.21.15.46:6789/0 conn(0x7f9584071cb0 legacy=0x7f958415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958415d660 con 0x7f9584153300 2023-10-30T12:57:38.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f957c001a50 con 0x7f9584153300 2023-10-30T12:57:38.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 >> v1:172.21.15.8:6789/0 conn(0x7f9584153300 legacy=0x7f95840cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9574003f30 0x7f95740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574004d10 0x7f9574005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840cca10 con 0x7f9574003f30 2023-10-30T12:57:38.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957400bb00 con 0x7f9574004d10 2023-10-30T12:57:38.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957400bcc0 con 0x7f95840718d0 2023-10-30T12:57:38.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f9583fff640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9574003f30 0x7f95740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95888da640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.050+0000 7f95890db640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574004d10 0x7f9574005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 -- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 msgr2=0x7f95740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f95840718d0 0x7f95740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 -- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574004d10 msgr2=0x7f9574005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574004d10 0x7f9574005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 -- 172.21.15.8:0/3571617418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574003530 con 0x7f9574003f30 2023-10-30T12:57:38.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95890db640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574004d10 0x7f9574005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f9583fff640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9574003f30 0x7f95740047d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9570001750 tx=0x7f957001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4776d93d7986942e server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95700052d0 con 0x7f9574003f30 2023-10-30T12:57:38.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700a7050 con 0x7f9574003f30 2023-10-30T12:57:38.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95837fe640 1 -- 172.21.15.8:0/3571617418 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570003600 con 0x7f9574003f30 2023-10-30T12:57:38.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 -- 172.21.15.8:0/3571617418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9574003f30 msgr2=0x7f95740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 --2- 172.21.15.8:0/3571617418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9574003f30 0x7f95740047d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9570001750 tx=0x7f957001c3c0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95890db640 1 -- 172.21.15.8:0/3571617418 reap_dead start 2023-10-30T12:57:38.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 -- 172.21.15.8:0/3571617418 shutdown_connections 2023-10-30T12:57:38.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 -- 172.21.15.8:0/3571617418 >> 172.21.15.8:0/3571617418 conn(0x7f958406ab00 msgr2=0x7f9584065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 -- 172.21.15.8:0/3571617418 shutdown_connections 2023-10-30T12:57:38.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.054+0000 7f95898dc640 1 -- 172.21.15.8:0/3571617418 wait complete. 2023-10-30T12:57:38.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 Processor -- start 2023-10-30T12:57:38.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 -- start start 2023-10-30T12:57:38.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 0x7f958415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f958415c130 0x7f9584142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415b5a0 con 0x7f95841f2c80 2023-10-30T12:57:38.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b38c0 con 0x7f9574003f30 2023-10-30T12:57:38.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95898dc640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b3b60 con 0x7f958415c130 2023-10-30T12:57:38.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44990/0 (socket says 172.21.15.8:44990) 2023-10-30T12:57:38.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 -- 172.21.15.8:0/4000464354 learned_addr learned my addr 172.21.15.8:0/4000464354 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:38.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f9583fff640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f958415c130 0x7f9584142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95888da640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 0x7f958415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 -- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f958415c130 msgr2=0x7f9584142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f958415c130 0x7f9584142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 -- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 msgr2=0x7f958415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 0x7f958415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 -- 172.21.15.8:0/4000464354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840b3e40 con 0x7f95841f2c80 2023-10-30T12:57:38.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95888da640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 0x7f958415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.058+0000 7f95890db640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f957c0013d0 tx=0x7f957c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c007b40 con 0x7f95841f2c80 2023-10-30T12:57:38.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c00ed20 con 0x7f95841f2c80 2023-10-30T12:57:38.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c013640 con 0x7f95841f2c80 2023-10-30T12:57:38.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95840b4120 con 0x7f95841f2c80 2023-10-30T12:57:38.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95841f2480 con 0x7f95841f2c80 2023-10-30T12:57:38.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958404e170 con 0x7f95841f2c80 2023-10-30T12:57:38.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f957c0137a0 con 0x7f95841f2c80 2023-10-30T12:57:38.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f957c084660 con 0x7f95841f2c80 2023-10-30T12:57:38.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95888da640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.062+0000 7f95888da640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 0x7f9568068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9578009be0 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.066+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f957c0552c0 con 0x7f95841f2c80 2023-10-30T12:57:38.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.226+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9584146780 con 0x7f95841f2c80 2023-10-30T12:57:38.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.226+0000 7f95817fa640 1 -- 172.21.15.8:0/4000464354 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f957c054e50 con 0x7f95841f2c80 2023-10-30T12:57:38.234 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-10-30T12:57:38.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 msgr2=0x7f9568068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 0x7f9568068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9578009be0 tx=0x7f9578008040 comp rx=0 tx=0).stop 2023-10-30T12:57:38.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 msgr2=0x7f9584147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f957c0013d0 tx=0x7f957c00e530 comp rx=0 tx=0).stop 2023-10-30T12:57:38.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 shutdown_connections 2023-10-30T12:57:38.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f95680664a0 0x7f9568068960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95841f2c80 0x7f9584147d20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f958415c130 0x7f9584142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 --2- 172.21.15.8:0/4000464354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9574003f30 0x7f958415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 >> 172.21.15.8:0/4000464354 conn(0x7f958406ab00 msgr2=0x7f95840733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 shutdown_connections 2023-10-30T12:57:38.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.230+0000 7f95898dc640 1 -- 172.21.15.8:0/4000464354 wait complete. 2023-10-30T12:57:38.255 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-30T12:57:38.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.394+0000 7f72c2495640 1 Processor -- start 2023-10-30T12:57:38.401 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.394+0000 7f72c2495640 1 -- start start 2023-10-30T12:57:38.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c2495640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72bc153a10 0x7f72bc0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c2495640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc15d0a0 con 0x7f72bc0718d0 2023-10-30T12:57:38.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c2495640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc15d380 con 0x7f72bc071cb0 2023-10-30T12:57:38.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c2495640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0cca10 con 0x7f72bc153a10 2023-10-30T12:57:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1493640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f72bc0718d0 0x7f72bc15a230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42748/0 (socket says 172.21.15.8:42748) 2023-10-30T12:57:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1493640 1 -- 172.21.15.8:0/4129967434 learned_addr learned my addr 172.21.15.8:0/4129967434 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:38.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1c94640 1 --2- 172.21.15.8:0/4129967434 >> v2:172.21.15.46:3300/0 conn(0x7f72bc153a10 0x7f72bc0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978365270 0 0) 0x7f72bc15d0a0 con 0x7f72bc0718d0 2023-10-30T12:57:38.405 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a8003530 con 0x7f72bc0718d0 2023-10-30T12:57:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1653391012 0 0) 0x7f72a8003530 con 0x7f72bc0718d0 2023-10-30T12:57:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 >> v2:172.21.15.46:3300/0 conn(0x7f72bc153a10 msgr2=0x7f72bc0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 --2- 172.21.15.8:0/4129967434 >> v2:172.21.15.46:3300/0 conn(0x7f72bc153a10 0x7f72bc0c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 >> v1:172.21.15.46:6789/0 conn(0x7f72bc071cb0 legacy=0x7f72bc153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc15d660 con 0x7f72bc0718d0 2023-10-30T12:57:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f72b8001680 con 0x7f72bc0718d0 2023-10-30T12:57:38.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 >> v1:172.21.15.8:6789/0 conn(0x7f72bc0718d0 legacy=0x7f72bc15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8003980 0x7f72a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72a8005190 0x7f72a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0cca10 con 0x7f72a80047d0 2023-10-30T12:57:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a800c040 con 0x7f72a8003980 2023-10-30T12:57:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a800c200 con 0x7f72a8005190 2023-10-30T12:57:38.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1493640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8003980 0x7f72a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1c94640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72a8005190 0x7f72a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 -- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72a8005190 msgr2=0x7f72a800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72a8005190 0x7f72a800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 -- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8003980 msgr2=0x7f72a8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8003980 0x7f72a8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c0c92640 1 -- 172.21.15.8:0/4129967434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8003530 con 0x7f72a80047d0 2023-10-30T12:57:38.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1c94640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72a8005190 0x7f72a800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1493640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8003980 0x7f72a8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.398+0000 7f72c1c94640 1 -- 172.21.15.8:0/4129967434 reap_dead start 2023-10-30T12:57:38.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c0c92640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72a8004c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001780 tx=0x7f72ac01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7492bc6148d21881 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0052d0 con 0x7f72a80047d0 2023-10-30T12:57:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0a7050 con 0x7f72a80047d0 2023-10-30T12:57:38.414 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72b3fff640 1 -- 172.21.15.8:0/4129967434 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac003600 con 0x7f72a80047d0 2023-10-30T12:57:38.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 -- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 msgr2=0x7f72a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72a8004c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001780 tx=0x7f72ac01c3e0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 -- 172.21.15.8:0/4129967434 shutdown_connections 2023-10-30T12:57:38.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 --2- 172.21.15.8:0/4129967434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72a8004c50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 -- 172.21.15.8:0/4129967434 >> 172.21.15.8:0/4129967434 conn(0x7f72bc06ab00 msgr2=0x7f72bc065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 -- 172.21.15.8:0/4129967434 shutdown_connections 2023-10-30T12:57:38.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 -- 172.21.15.8:0/4129967434 wait complete. 2023-10-30T12:57:38.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.402+0000 7f72c2495640 1 Processor -- start 2023-10-30T12:57:38.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- start start 2023-10-30T12:57:38.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72bc15a970 0x7f72bc158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 0x7f72bc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc14f360 con 0x7f72a80047d0 2023-10-30T12:57:38.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1f3050 con 0x7f72bc0b3270 2023-10-30T12:57:38.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0b2480 con 0x7f72bc15a970 2023-10-30T12:57:38.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45010/0 (socket says 172.21.15.8:45010) 2023-10-30T12:57:38.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 -- 172.21.15.8:0/2911448088 learned_addr learned my addr 172.21.15.8:0/2911448088 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:38.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c0c92640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72bc15a970 0x7f72bc158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1c94640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 0x7f72bc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 -- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72bc15a970 msgr2=0x7f72bc158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72bc15a970 0x7f72bc158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 -- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 msgr2=0x7f72bc0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 0x7f72bc0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 -- 172.21.15.8:0/2911448088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0b29b0 con 0x7f72a80047d0 2023-10-30T12:57:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1c94640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 0x7f72bc0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c1493640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e780 tx=0x7f72b800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ee90 con 0x7f72a80047d0 2023-10-30T12:57:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800c8d0 con 0x7f72a80047d0 2023-10-30T12:57:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800cc40 con 0x7f72a80047d0 2023-10-30T12:57:38.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc0b26d0 con 0x7f72a80047d0 2023-10-30T12:57:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc14eca0 con 0x7f72a80047d0 2023-10-30T12:57:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.406+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc04e170 con 0x7f72a80047d0 2023-10-30T12:57:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.410+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f72b8013420 con 0x7f72a80047d0 2023-10-30T12:57:38.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.410+0000 7f72b1ffb640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 0x7f72a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.410+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f72b8083df0 con 0x7f72a80047d0 2023-10-30T12:57:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.410+0000 7f72c0c92640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 0x7f72a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.410+0000 7f72c0c92640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 0x7f72a0068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001750 tx=0x7f72ac0011a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.414+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f72b80549c0 con 0x7f72a80047d0 2023-10-30T12:57:38.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.570+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72bc0c9f80 con 0x7f72a80047d0 2023-10-30T12:57:38.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.570+0000 7f72b1ffb640 1 -- 172.21.15.8:0/2911448088 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72b8054550 con 0x7f72a80047d0 2023-10-30T12:57:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-10-30T12:57:38.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 msgr2=0x7f72a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 0x7f72a0068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001750 tx=0x7f72ac0011a0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 msgr2=0x7f72bc1f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e780 tx=0x7f72b800ec40 comp rx=0 tx=0).stop 2023-10-30T12:57:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 shutdown_connections 2023-10-30T12:57:38.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f72a00664d0 0x7f72a0068990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc0b3270 0x7f72bc0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f72bc15a970 0x7f72bc158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 --2- 172.21.15.8:0/2911448088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72a80047d0 0x7f72bc1f2b10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 >> 172.21.15.8:0/2911448088 conn(0x7f72bc06ab00 msgr2=0x7f72bc0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 shutdown_connections 2023-10-30T12:57:38.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.574+0000 7f72c2495640 1 -- 172.21.15.8:0/2911448088 wait complete. 2023-10-30T12:57:38.599 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-30T12:57:38.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.738+0000 7f8949e46640 1 Processor -- start 2023-10-30T12:57:38.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 -- start start 2023-10-30T12:57:38.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f894406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8944157290 0x7f894406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8944070360 con 0x7f89440c8330 2023-10-30T12:57:38.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894406f4f0 con 0x7f8944157290 2023-10-30T12:57:38.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8949e46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894406b2e0 con 0x7f8944156eb0 2023-10-30T12:57:38.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8944157290 0x7f894406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8944157290 0x7f894406efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45024/0 (socket says 172.21.15.8:45024) 2023-10-30T12:57:38.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 -- 172.21.15.8:0/2474999573 learned_addr learned my addr 172.21.15.8:0/2474999573 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:38.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f89437fe640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f894406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 -- 172.21.15.8:0/2474999573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 msgr2=0x7f894406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:38.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f894406cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 -- 172.21.15.8:0/2474999573 >> v1:172.21.15.46:6789/0 conn(0x7f89440c8330 legacy=0x7f89440c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.742+0000 7f8942ffd640 1 -- 172.21.15.8:0/2474999573 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894406fac0 con 0x7f8944157290 2023-10-30T12:57:38.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8942ffd640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8944157290 0x7f894406efb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8938009870 tx=0x7f893801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec0aa55dadea36d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f89427fc640 1 -- 172.21.15.8:0/2474999573 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380ab650 con 0x7f8944157290 2023-10-30T12:57:38.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f89427fc640 1 -- 172.21.15.8:0/2474999573 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89380ab7b0 con 0x7f8944157290 2023-10-30T12:57:38.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f89427fc640 1 -- 172.21.15.8:0/2474999573 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380abb40 con 0x7f8944157290 2023-10-30T12:57:38.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 -- 172.21.15.8:0/2474999573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8944157290 msgr2=0x7f894406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8944157290 0x7f894406efb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8938009870 tx=0x7f893801c540 comp rx=0 tx=0).stop 2023-10-30T12:57:38.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 -- 172.21.15.8:0/2474999573 shutdown_connections 2023-10-30T12:57:38.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8944157290 0x7f894406efb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 --2- 172.21.15.8:0/2474999573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f894406cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 -- 172.21.15.8:0/2474999573 >> 172.21.15.8:0/2474999573 conn(0x7f89440723f0 msgr2=0x7f8944072800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 -- 172.21.15.8:0/2474999573 shutdown_connections 2023-10-30T12:57:38.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 -- 172.21.15.8:0/2474999573 wait complete. 2023-10-30T12:57:38.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.746+0000 7f8949e46640 1 Processor -- start 2023-10-30T12:57:38.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- start start 2023-10-30T12:57:38.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f8944158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 0x7f8944142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8944147df0 con 0x7f89440c8330 2023-10-30T12:57:38.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440cc0f0 con 0x7f8944157290 2023-10-30T12:57:38.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89440cd4a0 con 0x7f8944156eb0 2023-10-30T12:57:38.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45026/0 (socket says 172.21.15.8:45026) 2023-10-30T12:57:38.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 -- 172.21.15.8:0/597052029 learned_addr learned my addr 172.21.15.8:0/597052029 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:38.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8942ffd640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f8944158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8943fff640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 0x7f8944142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 -- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 msgr2=0x7f8944158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f8944158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 -- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 msgr2=0x7f8944142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 0x7f8944142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 -- 172.21.15.8:0/597052029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380094d0 con 0x7f89440c8330 2023-10-30T12:57:38.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8943fff640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 0x7f8944142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:38.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f89437fe640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f893400e960 tx=0x7f893400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934009690 con 0x7f89440c8330 2023-10-30T12:57:38.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893400ca00 con 0x7f89440c8330 2023-10-30T12:57:38.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400cd70 con 0x7f89440c8330 2023-10-30T12:57:38.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8944147830 con 0x7f89440c8330 2023-10-30T12:57:38.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89440cbaa0 con 0x7f89440c8330 2023-10-30T12:57:38.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.750+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89440cb900 con 0x7f89440c8330 2023-10-30T12:57:38.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.754+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8934013470 con 0x7f89440c8330 2023-10-30T12:57:38.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.754+0000 7f892bfff640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 0x7f8930068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:38.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.754+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f8934084430 con 0x7f89440c8330 2023-10-30T12:57:38.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.754+0000 7f8942ffd640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 0x7f8930068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:38.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.754+0000 7f8942ffd640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 0x7f8930068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f89380af000 tx=0x7f89380b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:38.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.758+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89440cb900 con 0x7f89440c8330 2023-10-30T12:57:38.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.914+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89440cb900 con 0x7f89440c8330 2023-10-30T12:57:38.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.914+0000 7f892bfff640 1 -- 172.21.15.8:0/597052029 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89440cb900 con 0x7f89440c8330 2023-10-30T12:57:38.922 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-10-30T12:57:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 msgr2=0x7f8930068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 0x7f8930068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f89380af000 tx=0x7f89380b6040 comp rx=0 tx=0).stop 2023-10-30T12:57:38.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 msgr2=0x7f89440ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f893400e960 tx=0x7f893400ee20 comp rx=0 tx=0).stop 2023-10-30T12:57:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 shutdown_connections 2023-10-30T12:57:38.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f89300664a0 0x7f8930068960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8944157290 0x7f8944142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8944156eb0 0x7f8944158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 --2- 172.21.15.8:0/597052029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f89440c8330 0x7f89440ccf60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:38.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 >> 172.21.15.8:0/597052029 conn(0x7f89440723f0 msgr2=0x7f89440727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 shutdown_connections 2023-10-30T12:57:38.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:38.918+0000 7f8949e46640 1 -- 172.21.15.8:0/597052029 wait complete. 2023-10-30T12:57:38.943 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-30T12:57:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 Processor -- start 2023-10-30T12:57:39.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 -- start start 2023-10-30T12:57:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f22f80ca0d0 0x7f22f81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f80c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22f80c5560 0x7f22f8153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f80b27b0 con 0x7f22f80ca0d0 2023-10-30T12:57:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f80c4b10 con 0x7f22f80c5560 2023-10-30T12:57:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fe1b5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80c50d0 con 0x7f22f80ca4b0 2023-10-30T12:57:39.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f22f80ca0d0 0x7f22f81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f22f80ca0d0 0x7f22f81596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45032/0 (socket says 172.21.15.8:45032) 2023-10-30T12:57:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 -- 172.21.15.8:0/777095873 learned_addr learned my addr 172.21.15.8:0/777095873 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f80c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd9b4640 1 --2- 172.21.15.8:0/777095873 >> v2:172.21.15.46:3300/0 conn(0x7f22f80c5560 0x7f22f8153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 -- 172.21.15.8:0/777095873 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 msgr2=0x7f22f80c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f80c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 -- 172.21.15.8:0/777095873 >> v2:172.21.15.46:3300/0 conn(0x7f22f80c5560 msgr2=0x7f22f8153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:39.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/777095873 >> v2:172.21.15.46:3300/0 conn(0x7f22f80c5560 0x7f22f8153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fd1b3640 1 -- 172.21.15.8:0/777095873 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8159f80 con 0x7f22f80ca0d0 2023-10-30T12:57:39.093 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.082+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f80c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:39.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80ca0d0 0x7f22f81596f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f22f80c4000 tx=0x7f22f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effa4800f1351b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:39.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22effff640 1 -- 172.21.15.8:0/777095873 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40aa5f0 con 0x7f22f80ca0d0 2023-10-30T12:57:39.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22effff640 1 -- 172.21.15.8:0/777095873 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40aa750 con 0x7f22f80ca0d0 2023-10-30T12:57:39.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22effff640 1 -- 172.21.15.8:0/777095873 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40aaae0 con 0x7f22f80ca0d0 2023-10-30T12:57:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 -- 172.21.15.8:0/777095873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80ca0d0 msgr2=0x7f22f81596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80ca0d0 0x7f22f81596f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f22f80c4000 tx=0x7f22f401c4d0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 -- 172.21.15.8:0/777095873 shutdown_connections 2023-10-30T12:57:39.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/777095873 >> v2:172.21.15.46:3300/0 conn(0x7f22f80c5560 0x7f22f8153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f80c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/777095873 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80ca0d0 0x7f22f81596f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 -- 172.21.15.8:0/777095873 >> 172.21.15.8:0/777095873 conn(0x7f22f806bf20 msgr2=0x7f22f806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 -- 172.21.15.8:0/777095873 shutdown_connections 2023-10-30T12:57:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.086+0000 7f22fe1b5640 1 -- 172.21.15.8:0/777095873 wait complete. 2023-10-30T12:57:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 Processor -- start 2023-10-30T12:57:39.098 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- start start 2023-10-30T12:57:39.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 0x7f22f80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80b3bc0 con 0x7f22f80c5560 2023-10-30T12:57:39.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f815c8e0 con 0x7f22f80ca0d0 2023-10-30T12:57:39.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f815bf10 con 0x7f22f80ca4b0 2023-10-30T12:57:39.102 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45042/0 (socket says 172.21.15.8:45042) 2023-10-30T12:57:39.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 -- 172.21.15.8:0/4162988118 learned_addr learned my addr 172.21.15.8:0/4162988118 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:39.103 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 0x7f22f80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd9b4640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 -- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 msgr2=0x7f22f81581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:39.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f81581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 -- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 msgr2=0x7f22f80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:39.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 0x7f22f80b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 -- 172.21.15.8:0/4162988118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40094d0 con 0x7f22f80c5560 2023-10-30T12:57:39.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 0x7f22f80b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:57:39.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd9b4640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f81581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:57:39.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fd1b3640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f22f40ac040 tx=0x7f22f401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:39.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40b3800 con 0x7f22f80c5560 2023-10-30T12:57:39.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40b3960 con 0x7f22f80c5560 2023-10-30T12:57:39.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f8157810 con 0x7f22f80c5560 2023-10-30T12:57:39.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40b3cf0 con 0x7f22f80c5560 2023-10-30T12:57:39.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.090+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f8153f90 con 0x7f22f80c5560 2023-10-30T12:57:39.107 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f80cb130 con 0x7f22f80c5560 2023-10-30T12:57:39.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f22f40c6390 con 0x7f22f80c5560 2023-10-30T12:57:39.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22edffb640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 0x7f22dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:39.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f22f40a7070 con 0x7f22f80c5560 2023-10-30T12:57:39.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 0x7f22dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:39.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.094+0000 7f22fc9b2640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 0x7f22dc068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22f80d0020 tx=0x7f22e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:39.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.098+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f22f40ebd30 con 0x7f22f80c5560 2023-10-30T12:57:39.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.266+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f22f8067540 con 0x7f22f80c5560 2023-10-30T12:57:39.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.266+0000 7f22edffb640 1 -- 172.21.15.8:0/4162988118 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v53) v1 ==== 81+0+720 (secure 0 0 0) 0x7f22f40ef350 con 0x7f22f80c5560 2023-10-30T12:57:39.274 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6838/3033347051 2023-10-31T12:57:07.233391+0000 2023-10-30T12:57:39.274 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6839/3916953811 2023-10-31T12:55:12.201822+0000 2023-10-30T12:57:39.275 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6837/2926864154 2023-10-31T12:53:02.155598+0000 2023-10-30T12:57:39.275 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6836/3916953811 2023-10-31T12:55:12.201822+0000 2023-10-30T12:57:39.275 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.46:6833/3888673612 2023-10-31T12:50:57.121966+0000 2023-10-30T12:57:39.275 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6835/3033347051 2023-10-31T12:57:07.233391+0000 2023-10-30T12:57:39.276 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.46:6832/3888673612 2023-10-31T12:50:57.121966+0000 2023-10-30T12:57:39.276 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6835/2992596118 2023-10-31T12:49:27.084924+0000 2023-10-30T12:57:39.276 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6838/2992596118 2023-10-31T12:49:27.084924+0000 2023-10-30T12:57:39.277 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6839/710307090 2023-10-31T12:47:22.043669+0000 2023-10-30T12:57:39.277 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6834/2926864154 2023-10-31T12:53:02.155598+0000 2023-10-30T12:57:39.277 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6836/710307090 2023-10-31T12:47:22.043669+0000 2023-10-30T12:57:39.279 INFO:teuthology.orchestra.run.smithi008.stderr:listed 12 entries 2023-10-30T12:57:39.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 msgr2=0x7f22dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:39.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 0x7f22dc068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22f80d0020 tx=0x7f22e8008040 comp rx=0 tx=0).stop 2023-10-30T12:57:39.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 msgr2=0x7f22f80b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:39.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f22f40ac040 tx=0x7f22f401ad10 comp rx=0 tx=0).stop 2023-10-30T12:57:39.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 shutdown_connections 2023-10-30T12:57:39.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f22dc0664a0 0x7f22dc068960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f80ca0d0 0x7f22f80b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f22f80ca4b0 0x7f22f81581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 --2- 172.21.15.8:0/4162988118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22f80c5560 0x7f22f80b4010 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:39.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 >> 172.21.15.8:0/4162988118 conn(0x7f22f806bf20 msgr2=0x7f22f80cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:39.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 shutdown_connections 2023-10-30T12:57:39.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:39.270+0000 7f22fe1b5640 1 -- 172.21.15.8:0/4162988118 wait complete. 2023-10-30T12:57:39.294 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-30T12:57:39.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:57:39.295 DEBUG:teuthology.orchestra.run.smithi008:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-30T12:57:39.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:57:39.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:57:39.664 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-30T12:57:39.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:57:39.676 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-10-30T12:57:39.727 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-30T12:57:39.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:57:39.728 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-30T12:57:39.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-30T12:57:39.792 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2023-10-30T12:57:40.004 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-30T12:57:40.014 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T12:57:40.015 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:40.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 Processor -- start 2023-10-30T12:57:40.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 -- start start 2023-10-30T12:57:40.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb004070520 0x7fb004070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040c2310 con 0x7fb0040cb7d0 2023-10-30T12:57:40.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb004156710 con 0x7fb004150830 2023-10-30T12:57:40.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00ad72640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00406bc10 con 0x7fb004070520 2023-10-30T12:57:40.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb003fff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fb0040cb7d0 0x7fb0040c3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37930/0 (socket says 172.21.15.8:37930) 2023-10-30T12:57:40.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb003fff640 1 -- 172.21.15.8:0/847011353 learned_addr learned my addr 172.21.15.8:0/847011353 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:40.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> v2:172.21.15.46:3300/0 conn(0x7fb004070520 0x7fb004070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752514821 0 0) 0x7fb0040c2310 con 0x7fb0040cb7d0 2023-10-30T12:57:40.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff4003530 con 0x7fb0040cb7d0 2023-10-30T12:57:40.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118082644 0 0) 0x7faff4003530 con 0x7fb0040cb7d0 2023-10-30T12:57:40.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 >> v2:172.21.15.46:3300/0 conn(0x7fb004070520 msgr2=0x7fb004070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 --2- 172.21.15.8:0/847011353 >> v2:172.21.15.46:3300/0 conn(0x7fb004070520 0x7fb004070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 >> v1:172.21.15.46:6789/0 conn(0x7fb004150830 legacy=0x7fb00406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040c4360 con 0x7fb0040cb7d0 2023-10-30T12:57:40.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> v2:172.21.15.46:3300/0 conn(0x7fb004070520 0x7fb004070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:57:40.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7faff00016a0 con 0x7fb0040cb7d0 2023-10-30T12:57:40.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 >> v1:172.21.15.8:6789/0 conn(0x7fb0040cb7d0 legacy=0x7fb0040c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7faff4003980 0x7faff4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40047d0 0x7faff4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 0x7faff400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00406bc10 con 0x7faff4005190 2023-10-30T12:57:40.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff400c040 con 0x7faff40047d0 2023-10-30T12:57:40.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff400e410 con 0x7faff4003980 2023-10-30T12:57:40.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 0x7faff400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb003fff640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40047d0 0x7faff4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb008b0e640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7faff4003980 0x7faff4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 -- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7faff4003980 msgr2=0x7faff4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7faff4003980 0x7faff4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 -- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40047d0 msgr2=0x7faff4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40047d0 0x7faff4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.154+0000 7fb00930f640 1 -- 172.21.15.8:0/847011353 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4003530 con 0x7faff4005190 2023-10-30T12:57:40.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00930f640 1 -- 172.21.15.8:0/847011353 reap_dead start 2023-10-30T12:57:40.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00930f640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 0x7faff400b990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e350 tx=0x7faffc01ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36219318d43ebf server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0a5c70 con 0x7faff4005190 2023-10-30T12:57:40.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00f040 con 0x7faff4005190 2023-10-30T12:57:40.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb0037fe640 1 -- 172.21.15.8:0/847011353 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc003be0 con 0x7faff4005190 2023-10-30T12:57:40.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 -- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 msgr2=0x7faff400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 0x7faff400b990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e350 tx=0x7faffc01ce60 comp rx=0 tx=0).stop 2023-10-30T12:57:40.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 -- 172.21.15.8:0/847011353 shutdown_connections 2023-10-30T12:57:40.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 --2- 172.21.15.8:0/847011353 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faff4005190 0x7faff400b990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 -- 172.21.15.8:0/847011353 >> 172.21.15.8:0/847011353 conn(0x7fb0040cac70 msgr2=0x7fb0040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:40.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 -- 172.21.15.8:0/847011353 shutdown_connections 2023-10-30T12:57:40.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.158+0000 7fb00ad72640 1 -- 172.21.15.8:0/847011353 wait complete. 2023-10-30T12:57:40.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 Processor -- start 2023-10-30T12:57:40.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- start start 2023-10-30T12:57:40.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff4005190 0x7fb00415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb0041ea680 0x7fb0040ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040c9ed0 con 0x7fb0041577c0 2023-10-30T12:57:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040c9b80 con 0x7faff4005190 2023-10-30T12:57:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041597a0 con 0x7fb0041ea680 2023-10-30T12:57:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48090/0 (socket says 172.21.15.8:48090) 2023-10-30T12:57:40.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 -- 172.21.15.8:0/1139907304 learned_addr learned my addr 172.21.15.8:0/1139907304 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:40.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb008b0e640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff4005190 0x7fb00415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb003fff640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb0041ea680 0x7fb0040ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 -- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb0041ea680 msgr2=0x7fb0040ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb0041ea680 0x7fb0040ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 -- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff4005190 msgr2=0x7fb00415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff4005190 0x7fb00415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 -- 172.21.15.8:0/1139907304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb004159a80 con 0x7fb0041577c0 2023-10-30T12:57:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00930f640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7faffc01d700 tx=0x7faffc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0bd070 con 0x7fb0041577c0 2023-10-30T12:57:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc0039a0 con 0x7fb0041577c0 2023-10-30T12:57:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0a5c70 con 0x7fb0041577c0 2023-10-30T12:57:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb004159d60 con 0x7fb0041577c0 2023-10-30T12:57:40.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.162+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041594a0 con 0x7fb0041577c0 2023-10-30T12:57:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00404e170 con 0x7fb0041577c0 2023-10-30T12:57:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faffc0b8470 con 0x7fb0041577c0 2023-10-30T12:57:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb0017fa640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 0x7fafe8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7faffc11f4a0 con 0x7fb0041577c0 2023-10-30T12:57:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb008b0e640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 0x7fafe8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.166+0000 7fb008b0e640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 0x7fafe8068990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faff8000c10 tx=0x7faff8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.170+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faffc0ec670 con 0x7fb0041577c0 2023-10-30T12:57:40.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.338+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0040c9d40 con 0x7fafe80664d0 2023-10-30T12:57:40.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.346+0000 7fb0017fa640 1 -- 172.21.15.8:0/1139907304 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292338 (secure 0 0 0) 0x7fb0040c9d40 con 0x7fafe80664d0 2023-10-30T12:57:40.357 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:40.358 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:57:40.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 msgr2=0x7fafe8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 0x7fafe8068990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faff8000c10 tx=0x7faff8009040 comp rx=0 tx=0).stop 2023-10-30T12:57:40.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 msgr2=0x7fb0040c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7faffc01d700 tx=0x7faffc009750 comp rx=0 tx=0).stop 2023-10-30T12:57:40.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 shutdown_connections 2023-10-30T12:57:40.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fafe80664d0 0x7fafe8068990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0041577c0 0x7fb0040c5b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fb0041ea680 0x7fb0040ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 --2- 172.21.15.8:0/1139907304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff4005190 0x7fb00415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 >> 172.21.15.8:0/1139907304 conn(0x7fb0040cac70 msgr2=0x7fb0040c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:40.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 shutdown_connections 2023-10-30T12:57:40.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.350+0000 7fb00ad72640 1 -- 172.21.15.8:0/1139907304 wait complete. 2023-10-30T12:57:40.378 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2023-10-30T12:57:39.807154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1100554776,"num_objects":10461,"num_object_clones":0,"num_object_copies":20922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10461,"num_whiteouts":0,"num_read":2527,"num_read_kb":243523,"num_write":26395,"num_write_kb":1226421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26803,"ondisk_log_size":26803,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2693764,"kb_used_data":2202820,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":752280956,"statfs":{"total":773094113280,"available":770335698944,"internally_reserved":0,"allocated":2255687680,"data_stored":2199551815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":502648378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2581772,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":178,"num_write_kb":2634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"3.3f","version":"53'324","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574594+0000","last_change":"2023-10-30T12:52:38.167461+0000","last_active":"2023-10-30T12:57:21.574594+0000","last_peered":"2023-10-30T12:57:21.574594+0000","last_clean":"2023-10-30T12:57:21.574594+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:57:21.574594+0000","last_undegraded":"2023-10-30T12:57:21.574594+0000","last_fullsized":"2023-10-30T12:57:21.574594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:23.473692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'336","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602281+0000","last_change":"2023-10-30T12:52:38.158597+0000","last_active":"2023-10-30T12:57:39.602281+0000","last_peered":"2023-10-30T12:57:39.602281+0000","last_clean":"2023-10-30T12:57:39.602281+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:57:39.602281+0000","last_undegraded":"2023-10-30T12:57:39.602281+0000","last_fullsized":"2023-10-30T12:57:39.602281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:51:39.552812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'359","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617323+0000","last_change":"2023-10-30T12:52:38.161971+0000","last_active":"2023-10-30T12:57:39.617323+0000","last_peered":"2023-10-30T12:57:39.617323+0000","last_clean":"2023-10-30T12:57:39.617323+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:39.617323+0000","last_undegraded":"2023-10-30T12:57:39.617323+0000","last_fullsized":"2023-10-30T12:57:39.617323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:05.398156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'289","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.826281+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:19.826281+0000","last_peered":"2023-10-30T12:57:19.826281+0000","last_clean":"2023-10-30T12:57:19.826281+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:19.826281+0000","last_undegraded":"2023-10-30T12:57:19.826281+0000","last_fullsized":"2023-10-30T12:57:19.826281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:39.214543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571441+0000","last_change":"2023-10-30T12:52:38.166749+0000","last_active":"2023-10-30T12:57:21.571441+0000","last_peered":"2023-10-30T12:57:21.571441+0000","last_clean":"2023-10-30T12:57:21.571441+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:57:21.571441+0000","last_undegraded":"2023-10-30T12:57:21.571441+0000","last_fullsized":"2023-10-30T12:57:21.571441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:27.480500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28212871,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":331,"num_write_kb":27730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'323","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561161+0000","last_change":"2023-10-30T12:52:38.190432+0000","last_active":"2023-10-30T12:57:21.561161+0000","last_peered":"2023-10-30T12:57:21.561161+0000","last_clean":"2023-10-30T12:57:21.561161+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:57:21.561161+0000","last_undegraded":"2023-10-30T12:57:21.561161+0000","last_fullsized":"2023-10-30T12:57:21.561161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:12.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.562949+0000","last_change":"2023-10-30T12:52:38.164109+0000","last_active":"2023-10-30T12:57:21.562949+0000","last_peered":"2023-10-30T12:57:21.562949+0000","last_clean":"2023-10-30T12:57:21.562949+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:21.562949+0000","last_undegraded":"2023-10-30T12:57:21.562949+0000","last_fullsized":"2023-10-30T12:57:21.562949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:03:58.478040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23818883,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":355,"num_write_kb":23453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'316","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642976+0000","last_change":"2023-10-30T12:52:38.375236+0000","last_active":"2023-10-30T12:57:39.642976+0000","last_peered":"2023-10-30T12:57:39.642976+0000","last_clean":"2023-10-30T12:57:39.642976+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:57:39.642976+0000","last_undegraded":"2023-10-30T12:57:39.642976+0000","last_fullsized":"2023-10-30T12:57:39.642976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:34.799606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'326","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574849+0000","last_change":"2023-10-30T12:52:38.168173+0000","last_active":"2023-10-30T12:57:21.574849+0000","last_peered":"2023-10-30T12:57:21.574849+0000","last_clean":"2023-10-30T12:57:21.574849+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:57:21.574849+0000","last_undegraded":"2023-10-30T12:57:21.574849+0000","last_fullsized":"2023-10-30T12:57:21.574849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:17:54.844918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767246,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":326,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'339","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677329+0000","last_change":"2023-10-30T12:52:38.960608+0000","last_active":"2023-10-30T12:57:39.677329+0000","last_peered":"2023-10-30T12:57:39.677329+0000","last_clean":"2023-10-30T12:57:39.677329+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:57:39.677329+0000","last_undegraded":"2023-10-30T12:57:39.677329+0000","last_fullsized":"2023-10-30T12:57:39.677329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:35.999124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'337","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642756+0000","last_change":"2023-10-30T12:52:38.959629+0000","last_active":"2023-10-30T12:57:39.642756+0000","last_peered":"2023-10-30T12:57:39.642756+0000","last_clean":"2023-10-30T12:57:39.642756+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:57:39.642756+0000","last_undegraded":"2023-10-30T12:57:39.642756+0000","last_fullsized":"2023-10-30T12:57:39.642756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:11:54.888888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'312","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674922+0000","last_change":"2023-10-30T12:52:38.378139+0000","last_active":"2023-10-30T12:57:39.674922+0000","last_peered":"2023-10-30T12:57:39.674922+0000","last_clean":"2023-10-30T12:57:39.674922+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:39.674922+0000","last_undegraded":"2023-10-30T12:57:39.674922+0000","last_fullsized":"2023-10-30T12:57:39.674922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:34.995464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'314","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.636430+0000","last_change":"2023-10-30T12:52:38.374275+0000","last_active":"2023-10-30T12:57:39.636430+0000","last_peered":"2023-10-30T12:57:39.636430+0000","last_clean":"2023-10-30T12:57:39.636430+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:57:39.636430+0000","last_undegraded":"2023-10-30T12:57:39.636430+0000","last_fullsized":"2023-10-30T12:57:39.636430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:43:44.654314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'352","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621960+0000","last_change":"2023-10-30T12:52:38.161570+0000","last_active":"2023-10-30T12:57:39.621960+0000","last_peered":"2023-10-30T12:57:39.621960+0000","last_clean":"2023-10-30T12:57:39.621960+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:57:39.621960+0000","last_undegraded":"2023-10-30T12:57:39.621960+0000","last_fullsized":"2023-10-30T12:57:39.621960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:20.253861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'344","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660738+0000","last_change":"2023-10-30T12:52:38.157652+0000","last_active":"2023-10-30T12:57:39.660738+0000","last_peered":"2023-10-30T12:57:39.660738+0000","last_clean":"2023-10-30T12:57:39.660738+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:57:39.660738+0000","last_undegraded":"2023-10-30T12:57:39.660738+0000","last_fullsized":"2023-10-30T12:57:39.660738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:45:33.635954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'358","reported_seq":2481,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561366+0000","last_change":"2023-10-30T12:52:38.190778+0000","last_active":"2023-10-30T12:57:21.561366+0000","last_peered":"2023-10-30T12:57:21.561366+0000","last_clean":"2023-10-30T12:57:21.561366+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:57:21.561366+0000","last_undegraded":"2023-10-30T12:57:21.561366+0000","last_fullsized":"2023-10-30T12:57:21.561366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:23.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23735274,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":356,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.568060+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:21.568060+0000","last_peered":"2023-10-30T12:57:21.568060+0000","last_clean":"2023-10-30T12:57:21.568060+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:21.568060+0000","last_undegraded":"2023-10-30T12:57:21.568060+0000","last_fullsized":"2023-10-30T12:57:21.568060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:39.852314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10012551,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":322,"num_write_kb":14046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'346","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638116+0000","last_change":"2023-10-30T12:52:38.708281+0000","last_active":"2023-10-30T12:57:39.638116+0000","last_peered":"2023-10-30T12:57:39.638116+0000","last_clean":"2023-10-30T12:57:39.638116+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:57:39.638116+0000","last_undegraded":"2023-10-30T12:57:39.638116+0000","last_fullsized":"2023-10-30T12:57:39.638116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:24:56.978901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'341","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573140+0000","last_change":"2023-10-30T12:52:38.166885+0000","last_active":"2023-10-30T12:57:21.573140+0000","last_peered":"2023-10-30T12:57:21.573140+0000","last_clean":"2023-10-30T12:57:21.573140+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:57:21.573140+0000","last_undegraded":"2023-10-30T12:57:21.573140+0000","last_fullsized":"2023-10-30T12:57:21.573140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:17:23.403084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695056,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":340,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'351","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670046+0000","last_change":"2023-10-30T12:52:38.959805+0000","last_active":"2023-10-30T12:57:39.670046+0000","last_peered":"2023-10-30T12:57:39.670046+0000","last_clean":"2023-10-30T12:57:39.670046+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:57:39.670046+0000","last_undegraded":"2023-10-30T12:57:39.670046+0000","last_fullsized":"2023-10-30T12:57:39.670046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:45.750836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'355","reported_seq":2479,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671742+0000","last_change":"2023-10-30T12:52:38.159940+0000","last_active":"2023-10-30T12:57:39.671742+0000","last_peered":"2023-10-30T12:57:39.671742+0000","last_clean":"2023-10-30T12:57:39.671742+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:57:39.671742+0000","last_undegraded":"2023-10-30T12:57:39.671742+0000","last_fullsized":"2023-10-30T12:57:39.671742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:51:28.603483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'383","reported_seq":2506,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.648123+0000","last_change":"2023-10-30T12:52:38.707969+0000","last_active":"2023-10-30T12:57:39.648123+0000","last_peered":"2023-10-30T12:57:39.648123+0000","last_clean":"2023-10-30T12:57:39.648123+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:57:39.648123+0000","last_undegraded":"2023-10-30T12:57:39.648123+0000","last_fullsized":"2023-10-30T12:57:39.648123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:50.085992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'372","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659940+0000","last_change":"2023-10-30T12:52:38.374161+0000","last_active":"2023-10-30T12:57:39.659940+0000","last_peered":"2023-10-30T12:57:39.659940+0000","last_clean":"2023-10-30T12:57:39.659940+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:57:39.659940+0000","last_undegraded":"2023-10-30T12:57:39.659940+0000","last_fullsized":"2023-10-30T12:57:39.659940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:05.818986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'343","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561012+0000","last_change":"2023-10-30T12:52:38.192255+0000","last_active":"2023-10-30T12:57:21.561012+0000","last_peered":"2023-10-30T12:57:21.561012+0000","last_clean":"2023-10-30T12:57:21.561012+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:57:21.561012+0000","last_undegraded":"2023-10-30T12:57:21.561012+0000","last_fullsized":"2023-10-30T12:57:21.561012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:37.588996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'327","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658302+0000","last_change":"2023-10-30T12:52:38.374594+0000","last_active":"2023-10-30T12:57:39.658302+0000","last_peered":"2023-10-30T12:57:39.658302+0000","last_clean":"2023-10-30T12:57:39.658302+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:57:39.658302+0000","last_undegraded":"2023-10-30T12:57:39.658302+0000","last_fullsized":"2023-10-30T12:57:39.658302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:08.496852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'379","reported_seq":2510,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.669068+0000","last_change":"2023-10-30T12:52:38.960525+0000","last_active":"2023-10-30T12:57:39.669068+0000","last_peered":"2023-10-30T12:57:39.669068+0000","last_clean":"2023-10-30T12:57:39.669068+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:57:39.669068+0000","last_undegraded":"2023-10-30T12:57:39.669068+0000","last_fullsized":"2023-10-30T12:57:39.669068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:09.235440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617272+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:39.617272+0000","last_peered":"2023-10-30T12:57:39.617272+0000","last_clean":"2023-10-30T12:57:39.617272+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:39.617272+0000","last_undegraded":"2023-10-30T12:57:39.617272+0000","last_fullsized":"2023-10-30T12:57:39.617272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:38.878517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'306","reported_seq":2419,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670815+0000","last_change":"2023-10-30T12:52:38.375028+0000","last_active":"2023-10-30T12:57:39.670815+0000","last_peered":"2023-10-30T12:57:39.670815+0000","last_clean":"2023-10-30T12:57:39.670815+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:57:39.670815+0000","last_undegraded":"2023-10-30T12:57:39.670815+0000","last_fullsized":"2023-10-30T12:57:39.670815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:53.865022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'330","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.618194+0000","last_change":"2023-10-30T12:52:38.160968+0000","last_active":"2023-10-30T12:57:39.618194+0000","last_peered":"2023-10-30T12:57:39.618194+0000","last_clean":"2023-10-30T12:57:39.618194+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:39.618194+0000","last_undegraded":"2023-10-30T12:57:39.618194+0000","last_fullsized":"2023-10-30T12:57:39.618194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:14:14.802106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'360","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560965+0000","last_change":"2023-10-30T12:52:38.193081+0000","last_active":"2023-10-30T12:57:21.560965+0000","last_peered":"2023-10-30T12:57:21.560965+0000","last_clean":"2023-10-30T12:57:21.560965+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:57:21.560965+0000","last_undegraded":"2023-10-30T12:57:21.560965+0000","last_fullsized":"2023-10-30T12:57:21.560965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:19.937261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15312207,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":357,"num_write_kb":15140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'376","reported_seq":2497,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642772+0000","last_change":"2023-10-30T12:52:38.708132+0000","last_active":"2023-10-30T12:57:39.642772+0000","last_peered":"2023-10-30T12:57:39.642772+0000","last_clean":"2023-10-30T12:57:39.642772+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:57:39.642772+0000","last_undegraded":"2023-10-30T12:57:39.642772+0000","last_fullsized":"2023-10-30T12:57:39.642772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:23:07.870943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'336","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656156+0000","last_change":"2023-10-30T12:52:38.956531+0000","last_active":"2023-10-30T12:57:39.656156+0000","last_peered":"2023-10-30T12:57:39.656156+0000","last_clean":"2023-10-30T12:57:39.656156+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:57:39.656156+0000","last_undegraded":"2023-10-30T12:57:39.656156+0000","last_fullsized":"2023-10-30T12:57:39.656156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:37.995081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'342","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657965+0000","last_change":"2023-10-30T12:52:38.377927+0000","last_active":"2023-10-30T12:57:39.657965+0000","last_peered":"2023-10-30T12:57:39.657965+0000","last_clean":"2023-10-30T12:57:39.657965+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:39.657965+0000","last_undegraded":"2023-10-30T12:57:39.657965+0000","last_fullsized":"2023-10-30T12:57:39.657965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:29.073770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"36'262","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258065+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:57:12.258065+0000","last_peered":"2023-10-30T12:57:12.258065+0000","last_clean":"2023-10-30T12:57:12.258065+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:12.258065+0000","last_undegraded":"2023-10-30T12:57:12.258065+0000","last_fullsized":"2023-10-30T12:57:12.258065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'324","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657884+0000","last_change":"2023-10-30T12:52:38.161345+0000","last_active":"2023-10-30T12:57:39.657884+0000","last_peered":"2023-10-30T12:57:39.657884+0000","last_clean":"2023-10-30T12:57:39.657884+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:39.657884+0000","last_undegraded":"2023-10-30T12:57:39.657884+0000","last_fullsized":"2023-10-30T12:57:39.657884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:18.100051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'281","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.366158+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:57:39.366158+0000","last_peered":"2023-10-30T12:57:39.366158+0000","last_clean":"2023-10-30T12:57:39.366158+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:57:39.366158+0000","last_undegraded":"2023-10-30T12:57:39.366158+0000","last_fullsized":"2023-10-30T12:57:39.366158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403693,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":278,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"52'72","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242321+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:57:12.242321+0000","last_peered":"2023-10-30T12:57:12.242321+0000","last_clean":"2023-10-30T12:57:12.242321+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:12.242321+0000","last_undegraded":"2023-10-30T12:57:12.242321+0000","last_fullsized":"2023-10-30T12:57:12.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'400","reported_seq":2531,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.613041+0000","last_change":"2023-10-30T12:52:38.375331+0000","last_active":"2023-10-30T12:57:39.613041+0000","last_peered":"2023-10-30T12:57:39.613041+0000","last_clean":"2023-10-30T12:57:39.613041+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:39.613041+0000","last_undegraded":"2023-10-30T12:57:39.613041+0000","last_fullsized":"2023-10-30T12:57:39.613041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:29.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"52'341","reported_seq":392,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.262243+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:57:12.262243+0000","last_peered":"2023-10-30T12:57:12.262243+0000","last_clean":"2023-10-30T12:57:12.262243+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:12.262243+0000","last_undegraded":"2023-10-30T12:57:12.262243+0000","last_fullsized":"2023-10-30T12:57:12.262243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'356","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617014+0000","last_change":"2023-10-30T12:52:38.157348+0000","last_active":"2023-10-30T12:57:39.617014+0000","last_peered":"2023-10-30T12:57:39.617014+0000","last_clean":"2023-10-30T12:57:39.617014+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:57:39.617014+0000","last_undegraded":"2023-10-30T12:57:39.617014+0000","last_fullsized":"2023-10-30T12:57:39.617014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:06.624515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'31","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:29.786928+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:57:29.786928+0000","last_peered":"2023-10-30T12:57:29.786928+0000","last_clean":"2023-10-30T12:57:29.786928+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:29.786928+0000","last_undegraded":"2023-10-30T12:57:29.786928+0000","last_fullsized":"2023-10-30T12:57:29.786928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":31,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'350","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571444+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:21.571444+0000","last_peered":"2023-10-30T12:57:21.571444+0000","last_clean":"2023-10-30T12:57:21.571444+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:21.571444+0000","last_undegraded":"2023-10-30T12:57:21.571444+0000","last_fullsized":"2023-10-30T12:57:21.571444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:09.875012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:24.827512+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:57:24.827512+0000","last_peered":"2023-10-30T12:57:24.827512+0000","last_clean":"2023-10-30T12:57:24.827512+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:57:24.827512+0000","last_undegraded":"2023-10-30T12:57:24.827512+0000","last_fullsized":"2023-10-30T12:57:24.827512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069734,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":1048,"num_write":70,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'349","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.622444+0000","last_change":"2023-10-30T12:52:38.162408+0000","last_active":"2023-10-30T12:57:39.622444+0000","last_peered":"2023-10-30T12:57:39.622444+0000","last_clean":"2023-10-30T12:57:39.622444+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:57:39.622444+0000","last_undegraded":"2023-10-30T12:57:39.622444+0000","last_fullsized":"2023-10-30T12:57:39.622444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:14.339468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"52'43","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.246572+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:57:12.246572+0000","last_peered":"2023-10-30T12:57:12.246572+0000","last_clean":"2023-10-30T12:57:12.246572+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:12.246572+0000","last_undegraded":"2023-10-30T12:57:12.246572+0000","last_fullsized":"2023-10-30T12:57:12.246572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'332","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674677+0000","last_change":"2023-10-30T12:52:38.378836+0000","last_active":"2023-10-30T12:57:39.674677+0000","last_peered":"2023-10-30T12:57:39.674677+0000","last_clean":"2023-10-30T12:57:39.674677+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:39.674677+0000","last_undegraded":"2023-10-30T12:57:39.674677+0000","last_fullsized":"2023-10-30T12:57:39.674677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:57:03.390521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'107","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:31.407376+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:57:31.407376+0000","last_peered":"2023-10-30T12:57:31.407376+0000","last_clean":"2023-10-30T12:57:31.407376+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:31.407376+0000","last_undegraded":"2023-10-30T12:57:31.407376+0000","last_fullsized":"2023-10-30T12:57:31.407376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":106,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'306","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573471+0000","last_change":"2023-10-30T12:52:38.163983+0000","last_active":"2023-10-30T12:57:21.573471+0000","last_peered":"2023-10-30T12:57:21.573471+0000","last_clean":"2023-10-30T12:57:21.573471+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:57:21.573471+0000","last_undegraded":"2023-10-30T12:57:21.573471+0000","last_fullsized":"2023-10-30T12:57:21.573471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:26.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"52'354","reported_seq":416,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608375+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:57:07.608375+0000","last_peered":"2023-10-30T12:57:07.608375+0000","last_clean":"2023-10-30T12:57:07.608375+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:07.608375+0000","last_undegraded":"2023-10-30T12:57:07.608375+0000","last_fullsized":"2023-10-30T12:57:07.608375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'359","reported_seq":2485,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.661402+0000","last_change":"2023-10-30T12:52:38.161686+0000","last_active":"2023-10-30T12:57:39.661402+0000","last_peered":"2023-10-30T12:57:39.661402+0000","last_clean":"2023-10-30T12:57:39.661402+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:39.661402+0000","last_undegraded":"2023-10-30T12:57:39.661402+0000","last_fullsized":"2023-10-30T12:57:39.661402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:52.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"52'104","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.611180+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:57:07.611180+0000","last_peered":"2023-10-30T12:57:07.611180+0000","last_clean":"2023-10-30T12:57:07.611180+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:07.611180+0000","last_undegraded":"2023-10-30T12:57:07.611180+0000","last_fullsized":"2023-10-30T12:57:07.611180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":7385,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'345","reported_seq":2482,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571576+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:21.571576+0000","last_peered":"2023-10-30T12:57:21.571576+0000","last_clean":"2023-10-30T12:57:21.571576+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:21.571576+0000","last_undegraded":"2023-10-30T12:57:21.571576+0000","last_fullsized":"2023-10-30T12:57:21.571576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:51.506289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'394","reported_seq":453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.333732+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:57:39.333732+0000","last_peered":"2023-10-30T12:57:39.333732+0000","last_clean":"2023-10-30T12:57:39.333732+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:57:39.333732+0000","last_undegraded":"2023-10-30T12:57:39.333732+0000","last_fullsized":"2023-10-30T12:57:39.333732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16248710,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":387,"num_write_kb":16075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'330","reported_seq":2447,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656223+0000","last_change":"2023-10-30T12:52:38.959038+0000","last_active":"2023-10-30T12:57:39.656223+0000","last_peered":"2023-10-30T12:57:39.656223+0000","last_clean":"2023-10-30T12:57:39.656223+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:57:39.656223+0000","last_undegraded":"2023-10-30T12:57:39.656223+0000","last_fullsized":"2023-10-30T12:57:39.656223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:03:23.132047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"52'112","reported_seq":181,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608639+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:57:07.608639+0000","last_peered":"2023-10-30T12:57:07.608639+0000","last_clean":"2023-10-30T12:57:07.608639+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:07.608639+0000","last_undegraded":"2023-10-30T12:57:07.608639+0000","last_fullsized":"2023-10-30T12:57:07.608639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'357","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.739802+0000","last_change":"2023-10-30T12:52:38.707521+0000","last_active":"2023-10-30T12:57:39.739802+0000","last_peered":"2023-10-30T12:57:39.739802+0000","last_clean":"2023-10-30T12:57:39.739802+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:57:39.739802+0000","last_undegraded":"2023-10-30T12:57:39.739802+0000","last_fullsized":"2023-10-30T12:57:39.739802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:24.903914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'102","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:14.887635+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:57:14.887635+0000","last_peered":"2023-10-30T12:57:14.887635+0000","last_clean":"2023-10-30T12:57:14.887635+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:14.887635+0000","last_undegraded":"2023-10-30T12:57:14.887635+0000","last_fullsized":"2023-10-30T12:57:14.887635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":176,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'304","reported_seq":2424,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573853+0000","last_change":"2023-10-30T12:52:38.166069+0000","last_active":"2023-10-30T12:57:21.573853+0000","last_peered":"2023-10-30T12:57:21.573853+0000","last_clean":"2023-10-30T12:57:21.573853+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:57:21.573853+0000","last_undegraded":"2023-10-30T12:57:21.573853+0000","last_fullsized":"2023-10-30T12:57:21.573853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:36.932929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12071184,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":306,"num_write_kb":15946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"52'165","reported_seq":217,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745677+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:56:34.745677+0000","last_peered":"2023-10-30T12:56:34.745677+0000","last_clean":"2023-10-30T12:56:34.745677+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:56:34.745677+0000","last_undegraded":"2023-10-30T12:56:34.745677+0000","last_fullsized":"2023-10-30T12:56:34.745677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'319","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574457+0000","last_change":"2023-10-30T12:52:38.167701+0000","last_active":"2023-10-30T12:57:21.574457+0000","last_peered":"2023-10-30T12:57:21.574457+0000","last_clean":"2023-10-30T12:57:21.574457+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:57:21.574457+0000","last_undegraded":"2023-10-30T12:57:21.574457+0000","last_fullsized":"2023-10-30T12:57:21.574457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:54.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"52'123","reported_seq":174,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.787653+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:56:34.787653+0000","last_peered":"2023-10-30T12:56:34.787653+0000","last_clean":"2023-10-30T12:56:34.787653+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:56:34.787653+0000","last_undegraded":"2023-10-30T12:56:34.787653+0000","last_fullsized":"2023-10-30T12:56:34.787653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'357","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558014+0000","last_change":"2023-10-30T12:52:38.194662+0000","last_active":"2023-10-30T12:57:21.558014+0000","last_peered":"2023-10-30T12:57:21.558014+0000","last_clean":"2023-10-30T12:57:21.558014+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:57:21.558014+0000","last_undegraded":"2023-10-30T12:57:21.558014+0000","last_fullsized":"2023-10-30T12:57:21.558014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:33:24.813894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526551,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":357,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"52'12","reported_seq":70,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745646+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:56:34.745646+0000","last_peered":"2023-10-30T12:56:34.745646+0000","last_clean":"2023-10-30T12:56:34.745646+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:56:34.745646+0000","last_undegraded":"2023-10-30T12:56:34.745646+0000","last_fullsized":"2023-10-30T12:56:34.745646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.570633+0000","last_change":"2023-10-30T12:52:38.166263+0000","last_active":"2023-10-30T12:57:21.570633+0000","last_peered":"2023-10-30T12:57:21.570633+0000","last_clean":"2023-10-30T12:57:21.570633+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:57:21.570633+0000","last_undegraded":"2023-10-30T12:57:21.570633+0000","last_fullsized":"2023-10-30T12:57:21.570633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:44.813227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468332,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":322,"num_write_kb":15461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.243330+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:57:07.243330+0000","last_peered":"2023-10-30T12:57:07.243330+0000","last_clean":"2023-10-30T12:57:07.243330+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:07.243330+0000","last_undegraded":"2023-10-30T12:57:07.243330+0000","last_fullsized":"2023-10-30T12:57:07.243330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'10","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242909+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:57:12.242909+0000","last_peered":"2023-10-30T12:57:12.242909+0000","last_clean":"2023-10-30T12:57:12.242909+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:12.242909+0000","last_undegraded":"2023-10-30T12:57:12.242909+0000","last_fullsized":"2023-10-30T12:57:12.242909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'310","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.608876+0000","last_change":"2023-10-30T12:52:38.374620+0000","last_active":"2023-10-30T12:57:39.608876+0000","last_peered":"2023-10-30T12:57:39.608876+0000","last_clean":"2023-10-30T12:57:39.608876+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:57:39.608876+0000","last_undegraded":"2023-10-30T12:57:39.608876+0000","last_fullsized":"2023-10-30T12:57:39.608876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:36:22.529147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"52'497","reported_seq":557,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.260199+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:57:12.260199+0000","last_peered":"2023-10-30T12:57:12.260199+0000","last_clean":"2023-10-30T12:57:12.260199+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:12.260199+0000","last_undegraded":"2023-10-30T12:57:12.260199+0000","last_fullsized":"2023-10-30T12:57:12.260199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'327","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.663068+0000","last_change":"2023-10-30T12:52:38.159894+0000","last_active":"2023-10-30T12:57:39.663068+0000","last_peered":"2023-10-30T12:57:39.663068+0000","last_clean":"2023-10-30T12:57:39.663068+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:57:39.663068+0000","last_undegraded":"2023-10-30T12:57:39.663068+0000","last_fullsized":"2023-10-30T12:57:39.663068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:01.190957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'139","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258850+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:57:12.258850+0000","last_peered":"2023-10-30T12:57:12.258850+0000","last_clean":"2023-10-30T12:57:12.258850+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:57:12.258850+0000","last_undegraded":"2023-10-30T12:57:12.258850+0000","last_fullsized":"2023-10-30T12:57:12.258850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4302161,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":135,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'325","reported_seq":2452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617476+0000","last_change":"2023-10-30T12:52:38.157754+0000","last_active":"2023-10-30T12:57:39.617476+0000","last_peered":"2023-10-30T12:57:39.617476+0000","last_clean":"2023-10-30T12:57:39.617476+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:57:39.617476+0000","last_undegraded":"2023-10-30T12:57:39.617476+0000","last_fullsized":"2023-10-30T12:57:39.617476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:15:05.869155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"52'245","reported_seq":297,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.453118+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:56:35.453118+0000","last_peered":"2023-10-30T12:56:35.453118+0000","last_clean":"2023-10-30T12:56:35.453118+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:56:35.453118+0000","last_undegraded":"2023-10-30T12:56:35.453118+0000","last_fullsized":"2023-10-30T12:56:35.453118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'373","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.654586+0000","last_change":"2023-10-30T12:52:38.380527+0000","last_active":"2023-10-30T12:57:39.654586+0000","last_peered":"2023-10-30T12:57:39.654586+0000","last_clean":"2023-10-30T12:57:39.654586+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:39.654586+0000","last_undegraded":"2023-10-30T12:57:39.654586+0000","last_fullsized":"2023-10-30T12:57:39.654586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:28:52.581209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'3","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.239512+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:57:07.239512+0000","last_peered":"2023-10-30T12:57:07.239512+0000","last_clean":"2023-10-30T12:57:07.239512+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:57:07.239512+0000","last_undegraded":"2023-10-30T12:57:07.239512+0000","last_fullsized":"2023-10-30T12:57:07.239512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'315","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558514+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:21.558514+0000","last_peered":"2023-10-30T12:57:21.558514+0000","last_clean":"2023-10-30T12:57:21.558514+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:21.558514+0000","last_undegraded":"2023-10-30T12:57:21.558514+0000","last_fullsized":"2023-10-30T12:57:21.558514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:35.262822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9696989,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":311,"num_write_kb":13726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'9","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.243063+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:57:12.243063+0000","last_peered":"2023-10-30T12:57:12.243063+0000","last_clean":"2023-10-30T12:57:12.243063+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:12.243063+0000","last_undegraded":"2023-10-30T12:57:12.243063+0000","last_fullsized":"2023-10-30T12:57:12.243063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'341","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617288+0000","last_change":"2023-10-30T12:52:38.158164+0000","last_active":"2023-10-30T12:57:39.617288+0000","last_peered":"2023-10-30T12:57:39.617288+0000","last_clean":"2023-10-30T12:57:39.617288+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:57:39.617288+0000","last_undegraded":"2023-10-30T12:57:39.617288+0000","last_fullsized":"2023-10-30T12:57:39.617288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:42.652706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'99","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:26.427271+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:57:26.427271+0000","last_peered":"2023-10-30T12:57:26.427271+0000","last_clean":"2023-10-30T12:57:26.427271+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:57:26.427271+0000","last_undegraded":"2023-10-30T12:57:26.427271+0000","last_fullsized":"2023-10-30T12:57:26.427271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":238,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'314","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.655926+0000","last_change":"2023-10-30T12:52:38.374909+0000","last_active":"2023-10-30T12:57:39.655926+0000","last_peered":"2023-10-30T12:57:39.655926+0000","last_clean":"2023-10-30T12:57:39.655926+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:57:39.655926+0000","last_undegraded":"2023-10-30T12:57:39.655926+0000","last_fullsized":"2023-10-30T12:57:39.655926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:27:11.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"52'12","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610156+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:57:07.610156+0000","last_peered":"2023-10-30T12:57:07.610156+0000","last_clean":"2023-10-30T12:57:07.610156+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:07.610156+0000","last_undegraded":"2023-10-30T12:57:07.610156+0000","last_fullsized":"2023-10-30T12:57:07.610156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'331","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617128+0000","last_change":"2023-10-30T12:52:38.161992+0000","last_active":"2023-10-30T12:57:39.617128+0000","last_peered":"2023-10-30T12:57:39.617128+0000","last_clean":"2023-10-30T12:57:39.617128+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:39.617128+0000","last_undegraded":"2023-10-30T12:57:39.617128+0000","last_fullsized":"2023-10-30T12:57:39.617128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:36.903572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'163","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.279523+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:57:12.279523+0000","last_peered":"2023-10-30T12:57:12.279523+0000","last_clean":"2023-10-30T12:57:12.279523+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:12.279523+0000","last_undegraded":"2023-10-30T12:57:12.279523+0000","last_fullsized":"2023-10-30T12:57:12.279523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'340","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668698+0000","last_change":"2023-10-30T12:52:38.380106+0000","last_active":"2023-10-30T12:57:39.668698+0000","last_peered":"2023-10-30T12:57:39.668698+0000","last_clean":"2023-10-30T12:57:39.668698+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:39.668698+0000","last_undegraded":"2023-10-30T12:57:39.668698+0000","last_fullsized":"2023-10-30T12:57:39.668698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:12:33.391336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"52'58","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.324062+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:57:35.324062+0000","last_peered":"2023-10-30T12:57:35.324062+0000","last_clean":"2023-10-30T12:57:35.324062+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:57:35.324062+0000","last_undegraded":"2023-10-30T12:57:35.324062+0000","last_fullsized":"2023-10-30T12:57:35.324062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'327","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.579159+0000","last_change":"2023-10-30T12:52:38.707846+0000","last_active":"2023-10-30T12:57:39.579159+0000","last_peered":"2023-10-30T12:57:39.579159+0000","last_clean":"2023-10-30T12:57:39.579159+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:57:39.579159+0000","last_undegraded":"2023-10-30T12:57:39.579159+0000","last_fullsized":"2023-10-30T12:57:39.579159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:38:49.638944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'118","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.416434+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:57:19.416434+0000","last_peered":"2023-10-30T12:57:19.416434+0000","last_clean":"2023-10-30T12:57:19.416434+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:57:19.416434+0000","last_undegraded":"2023-10-30T12:57:19.416434+0000","last_fullsized":"2023-10-30T12:57:19.416434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":197,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'330","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602487+0000","last_change":"2023-10-30T12:52:38.373796+0000","last_active":"2023-10-30T12:57:39.602487+0000","last_peered":"2023-10-30T12:57:39.602487+0000","last_clean":"2023-10-30T12:57:39.602487+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:57:39.602487+0000","last_undegraded":"2023-10-30T12:57:39.602487+0000","last_fullsized":"2023-10-30T12:57:39.602487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:16.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"52'187","reported_seq":241,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.833028+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:56:34.833028+0000","last_peered":"2023-10-30T12:56:34.833028+0000","last_clean":"2023-10-30T12:56:34.833028+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:56:34.833028+0000","last_undegraded":"2023-10-30T12:56:34.833028+0000","last_fullsized":"2023-10-30T12:56:34.833028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":4104,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'332","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659101+0000","last_change":"2023-10-30T12:52:38.955966+0000","last_active":"2023-10-30T12:57:39.659101+0000","last_peered":"2023-10-30T12:57:39.659101+0000","last_clean":"2023-10-30T12:57:39.659101+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:57:39.659101+0000","last_undegraded":"2023-10-30T12:57:39.659101+0000","last_fullsized":"2023-10-30T12:57:39.659101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:55.980109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'314","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558152+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:21.558152+0000","last_peered":"2023-10-30T12:57:21.558152+0000","last_clean":"2023-10-30T12:57:21.558152+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:21.558152+0000","last_undegraded":"2023-10-30T12:57:21.558152+0000","last_fullsized":"2023-10-30T12:57:21.558152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:52.978943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22515664,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":311,"num_write_kb":26246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"52'151","reported_seq":202,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.983897+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:56:29.983897+0000","last_peered":"2023-10-30T12:56:29.983897+0000","last_clean":"2023-10-30T12:56:29.983897+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:56:29.983897+0000","last_undegraded":"2023-10-30T12:56:29.983897+0000","last_fullsized":"2023-10-30T12:56:29.983897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'315","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560674+0000","last_change":"2023-10-30T12:52:38.194210+0000","last_active":"2023-10-30T12:57:21.560674+0000","last_peered":"2023-10-30T12:57:21.560674+0000","last_clean":"2023-10-30T12:57:21.560674+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:57:21.560674+0000","last_undegraded":"2023-10-30T12:57:21.560674+0000","last_fullsized":"2023-10-30T12:57:21.560674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:56:10.108457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242644,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":313,"num_write_kb":14254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"52'362","reported_seq":420,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.158457+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:56:40.158457+0000","last_peered":"2023-10-30T12:56:40.158457+0000","last_clean":"2023-10-30T12:56:40.158457+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:56:40.158457+0000","last_undegraded":"2023-10-30T12:56:40.158457+0000","last_fullsized":"2023-10-30T12:56:40.158457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642959+0000","last_change":"2023-10-30T12:52:38.162371+0000","last_active":"2023-10-30T12:57:39.642959+0000","last_peered":"2023-10-30T12:57:39.642959+0000","last_clean":"2023-10-30T12:57:39.642959+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:57:39.642959+0000","last_undegraded":"2023-10-30T12:57:39.642959+0000","last_fullsized":"2023-10-30T12:57:39.642959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:05:58.391344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"52'278","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610628+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:57:07.610628+0000","last_peered":"2023-10-30T12:57:07.610628+0000","last_clean":"2023-10-30T12:57:07.610628+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:07.610628+0000","last_undegraded":"2023-10-30T12:57:07.610628+0000","last_fullsized":"2023-10-30T12:57:07.610628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'331","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638149+0000","last_change":"2023-10-30T12:52:38.373954+0000","last_active":"2023-10-30T12:57:39.638149+0000","last_peered":"2023-10-30T12:57:39.638149+0000","last_clean":"2023-10-30T12:57:39.638149+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:57:39.638149+0000","last_undegraded":"2023-10-30T12:57:39.638149+0000","last_fullsized":"2023-10-30T12:57:39.638149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:21.106315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"52'193","reported_seq":257,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.204972+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:56:14.204972+0000","last_peered":"2023-10-30T12:56:14.204972+0000","last_clean":"2023-10-30T12:56:14.204972+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:56:14.204972+0000","last_undegraded":"2023-10-30T12:56:14.204972+0000","last_fullsized":"2023-10-30T12:56:14.204972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925148655,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20870,"num_write_kb":1047645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1900888064,"data_stored":1847301778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21588,"ondisk_log_size":21588,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":174815753,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":580,"num_read_kb":143240,"num_write":5366,"num_write_kb":176360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350330880,"data_stored":350174933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5098,"ondisk_log_size":5098,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405832,"kb_used_data":326080,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93966008,"statfs":{"total":96636764160,"available":96221192192,"internally_reserved":0,"allocated":333905920,"data_stored":326089857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":81656267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.01}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":0.78400000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":1.272},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.073}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.399}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.327},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":1.4870000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281376,"kb_used_data":236312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090464,"statfs":{"total":96636764160,"available":96348635136,"internally_reserved":0,"allocated":241983488,"data_stored":236763651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.624}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.748}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.1319999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359668,"kb_used_data":292396,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94012172,"statfs":{"total":96636764160,"available":96268464128,"internally_reserved":0,"allocated":299413504,"data_stored":291703362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":1.046}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":1.121}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":1.147}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.008}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.75}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.2689999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416500,"kb_used_data":340908,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93955340,"statfs":{"total":96636764160,"available":96210268160,"internally_reserved":0,"allocated":349089792,"data_stored":340928408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":0.84299999999999997}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.753},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89700000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":0.999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.0529999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406132,"kb_used_data":334700,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93965708,"statfs":{"total":96636764160,"available":96220884992,"internally_reserved":0,"allocated":342732800,"data_stored":334429120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73136584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.60599999999999998,"5min":0.54100000000000004,"15min":0.499},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.786,"5min":1.867,"15min":1.867},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":176240,"kb_used_data":149224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94195600,"statfs":{"total":96636764160,"available":96456294400,"internally_reserved":0,"allocated":152805376,"data_stored":147769036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.496},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.68600000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266192,"kb_used_data":212808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94105648,"statfs":{"total":96636764160,"available":96364183552,"internally_reserved":0,"allocated":217915392,"data_stored":212359979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.92900000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":0.85199999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381824,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990016,"statfs":{"total":96636764160,"available":96245776384,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.374},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.45200000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12713984,"data_stored":12691497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42254336,"data_stored":42233706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":60223488,"data_stored":60206961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":34648064,"data_stored":34625544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49438720,"data_stored":49419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264155136,"data_stored":256775105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":261382144,"data_stored":253892880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:40.385 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:40.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.558+0000 7fa6b651d640 1 Processor -- start 2023-10-30T12:57:40.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.558+0000 7fa6b651d640 1 -- start start 2023-10-30T12:57:40.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b651d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa6b00cb7d0 0x7fa6b00c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b651d640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b00c2310 con 0x7fa6b0150830 2023-10-30T12:57:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b651d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b0156710 con 0x7fa6b00c3e40 2023-10-30T12:57:40.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b651d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b006bc10 con 0x7fa6b00cb7d0 2023-10-30T12:57:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6affff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa6b0150830 0x7fa6b006b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37946/0 (socket says 172.21.15.8:37946) 2023-10-30T12:57:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6affff640 1 -- 172.21.15.8:0/990035894 learned_addr learned my addr 172.21.15.8:0/990035894 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/990035894 >> v2:172.21.15.46:3300/0 conn(0x7fa6b00cb7d0 0x7fa6b00c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168270833 0 0) 0x7fa6b00c2310 con 0x7fa6b0150830 2023-10-30T12:57:40.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa698005740 con 0x7fa6b0150830 2023-10-30T12:57:40.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674314940 0 0) 0x7fa698005740 con 0x7fa6b0150830 2023-10-30T12:57:40.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 >> v2:172.21.15.46:3300/0 conn(0x7fa6b00cb7d0 msgr2=0x7fa6b00c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 --2- 172.21.15.8:0/990035894 >> v2:172.21.15.46:3300/0 conn(0x7fa6b00cb7d0 0x7fa6b00c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 >> v1:172.21.15.46:6789/0 conn(0x7fa6b00c3e40 legacy=0x7fa6b0070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00c4360 con 0x7fa6b0150830 2023-10-30T12:57:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7fa6a00016a0 con 0x7fa6b0150830 2023-10-30T12:57:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 >> v1:172.21.15.8:6789/0 conn(0x7fa6b0150830 legacy=0x7fa6b006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa698005b90 0x7fa698005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6980069e0 0x7fa698006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa69800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b006bc10 con 0x7fa6980073a0 2023-10-30T12:57:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69800e250 con 0x7fa6980069e0 2023-10-30T12:57:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69800e410 con 0x7fa698005b90 2023-10-30T12:57:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa69800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6980069e0 0x7fa698006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6affff640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa698005b90 0x7fa698005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 -- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa698005b90 msgr2=0x7fa698005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa698005b90 0x7fa698005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 -- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6980069e0 msgr2=0x7fa698006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6980069e0 0x7fa698006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6b4aba640 1 -- 172.21.15.8:0/990035894 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698005740 con 0x7fa6980073a0 2023-10-30T12:57:40.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.562+0000 7fa6affff640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa698005b90 0x7fa698005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:40.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b4aba640 1 -- 172.21.15.8:0/990035894 reap_dead start 2023-10-30T12:57:40.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b4aba640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa69800dba0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8001e40 tx=0x7fa6a801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5282ea16d7a2cae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8003b80 con 0x7fa6980073a0 2023-10-30T12:57:40.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8003ce0 con 0x7fa6980073a0 2023-10-30T12:57:40.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6aeffd640 1 -- 172.21.15.8:0/990035894 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8005260 con 0x7fa6980073a0 2023-10-30T12:57:40.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 msgr2=0x7fa69800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa69800dba0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8001e40 tx=0x7fa6a801c740 comp rx=0 tx=0).stop 2023-10-30T12:57:40.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- 172.21.15.8:0/990035894 shutdown_connections 2023-10-30T12:57:40.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 --2- 172.21.15.8:0/990035894 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa69800dba0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- 172.21.15.8:0/990035894 >> 172.21.15.8:0/990035894 conn(0x7fa6b00cac70 msgr2=0x7fa6b00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:40.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- 172.21.15.8:0/990035894 shutdown_connections 2023-10-30T12:57:40.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- 172.21.15.8:0/990035894 wait complete. 2023-10-30T12:57:40.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 Processor -- start 2023-10-30T12:57:40.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.566+0000 7fa6b651d640 1 -- start start 2023-10-30T12:57:40.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 0x7fa6b014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa6b01577c0 0x7fa6b0157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0158180 con 0x7fa6980073a0 2023-10-30T12:57:40.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b01596d0 con 0x7fa6b00c6020 2023-10-30T12:57:40.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0154750 con 0x7fa6b01577c0 2023-10-30T12:57:40.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48098/0 (socket says 172.21.15.8:48098) 2023-10-30T12:57:40.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 -- 172.21.15.8:0/293960083 learned_addr learned my addr 172.21.15.8:0/293960083 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:40.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 0x7fa6b014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b4aba640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa6b01577c0 0x7fa6b0157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 -- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa6b01577c0 msgr2=0x7fa6b0157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa6b01577c0 0x7fa6b0157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 -- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 msgr2=0x7fa6b014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 0x7fa6b014f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 -- 172.21.15.8:0/293960083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b01f1bc0 con 0x7fa6980073a0 2023-10-30T12:57:40.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 0x7fa6b014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:40.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6affff640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0001020 tx=0x7fa6a000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000ee40 con 0x7fa6980073a0 2023-10-30T12:57:40.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a000c8a0 con 0x7fa6980073a0 2023-10-30T12:57:40.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000cc30 con 0x7fa6980073a0 2023-10-30T12:57:40.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b01f1ea0 con 0x7fa6980073a0 2023-10-30T12:57:40.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b01f2460 con 0x7fa6980073a0 2023-10-30T12:57:40.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.570+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b0066510 con 0x7fa6980073a0 2023-10-30T12:57:40.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.574+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa6a001d460 con 0x7fa6980073a0 2023-10-30T12:57:40.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.574+0000 7fa6acff9640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 0x7fa69c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.574+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fa6a0084140 con 0x7fa6980073a0 2023-10-30T12:57:40.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.574+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 0x7fa69c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.574+0000 7fa6af7fe640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 0x7fa69c068990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0150000 tx=0x7fa6a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.578+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa6a0051280 con 0x7fa6980073a0 2023-10-30T12:57:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.730+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6b01569b0 con 0x7fa69c0664d0 2023-10-30T12:57:40.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.738+0000 7fa6acff9640 1 -- 172.21.15.8:0/293960083 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292338 (secure 0 0 0) 0x7fa6a40097d0 con 0x7fa69c0664d0 2023-10-30T12:57:40.748 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:40.750 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:57:40.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 msgr2=0x7fa69c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 0x7fa69c068990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0150000 tx=0x7fa6a4008040 comp rx=0 tx=0).stop 2023-10-30T12:57:40.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 msgr2=0x7fa6b00ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0001020 tx=0x7fa6a000ebf0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 shutdown_connections 2023-10-30T12:57:40.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fa69c0664d0 0x7fa69c068990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b00c6020 0x7fa6b014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fa6b01577c0 0x7fa6b0157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 --2- 172.21.15.8:0/293960083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6980073a0 0x7fa6b00ca6f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.754 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 >> 172.21.15.8:0/293960083 conn(0x7fa6b00cac70 msgr2=0x7fa6b0073740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 shutdown_connections 2023-10-30T12:57:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.742+0000 7fa6b651d640 1 -- 172.21.15.8:0/293960083 wait complete. 2023-10-30T12:57:40.770 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2023-10-30T12:57:39.807154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1100554776,"num_objects":10461,"num_object_clones":0,"num_object_copies":20922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10461,"num_whiteouts":0,"num_read":2527,"num_read_kb":243523,"num_write":26395,"num_write_kb":1226421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26803,"ondisk_log_size":26803,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2693764,"kb_used_data":2202820,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":752280956,"statfs":{"total":773094113280,"available":770335698944,"internally_reserved":0,"allocated":2255687680,"data_stored":2199551815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":502648378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2581772,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":178,"num_write_kb":2634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"3.3f","version":"53'324","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574594+0000","last_change":"2023-10-30T12:52:38.167461+0000","last_active":"2023-10-30T12:57:21.574594+0000","last_peered":"2023-10-30T12:57:21.574594+0000","last_clean":"2023-10-30T12:57:21.574594+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:57:21.574594+0000","last_undegraded":"2023-10-30T12:57:21.574594+0000","last_fullsized":"2023-10-30T12:57:21.574594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:23.473692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'336","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602281+0000","last_change":"2023-10-30T12:52:38.158597+0000","last_active":"2023-10-30T12:57:39.602281+0000","last_peered":"2023-10-30T12:57:39.602281+0000","last_clean":"2023-10-30T12:57:39.602281+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:57:39.602281+0000","last_undegraded":"2023-10-30T12:57:39.602281+0000","last_fullsized":"2023-10-30T12:57:39.602281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:51:39.552812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'359","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617323+0000","last_change":"2023-10-30T12:52:38.161971+0000","last_active":"2023-10-30T12:57:39.617323+0000","last_peered":"2023-10-30T12:57:39.617323+0000","last_clean":"2023-10-30T12:57:39.617323+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:39.617323+0000","last_undegraded":"2023-10-30T12:57:39.617323+0000","last_fullsized":"2023-10-30T12:57:39.617323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:05.398156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'289","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.826281+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:19.826281+0000","last_peered":"2023-10-30T12:57:19.826281+0000","last_clean":"2023-10-30T12:57:19.826281+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:19.826281+0000","last_undegraded":"2023-10-30T12:57:19.826281+0000","last_fullsized":"2023-10-30T12:57:19.826281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:39.214543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571441+0000","last_change":"2023-10-30T12:52:38.166749+0000","last_active":"2023-10-30T12:57:21.571441+0000","last_peered":"2023-10-30T12:57:21.571441+0000","last_clean":"2023-10-30T12:57:21.571441+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:57:21.571441+0000","last_undegraded":"2023-10-30T12:57:21.571441+0000","last_fullsized":"2023-10-30T12:57:21.571441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:27.480500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28212871,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":331,"num_write_kb":27730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'323","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561161+0000","last_change":"2023-10-30T12:52:38.190432+0000","last_active":"2023-10-30T12:57:21.561161+0000","last_peered":"2023-10-30T12:57:21.561161+0000","last_clean":"2023-10-30T12:57:21.561161+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:57:21.561161+0000","last_undegraded":"2023-10-30T12:57:21.561161+0000","last_fullsized":"2023-10-30T12:57:21.561161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:12.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.562949+0000","last_change":"2023-10-30T12:52:38.164109+0000","last_active":"2023-10-30T12:57:21.562949+0000","last_peered":"2023-10-30T12:57:21.562949+0000","last_clean":"2023-10-30T12:57:21.562949+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:21.562949+0000","last_undegraded":"2023-10-30T12:57:21.562949+0000","last_fullsized":"2023-10-30T12:57:21.562949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:03:58.478040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23818883,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":355,"num_write_kb":23453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'316","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642976+0000","last_change":"2023-10-30T12:52:38.375236+0000","last_active":"2023-10-30T12:57:39.642976+0000","last_peered":"2023-10-30T12:57:39.642976+0000","last_clean":"2023-10-30T12:57:39.642976+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:57:39.642976+0000","last_undegraded":"2023-10-30T12:57:39.642976+0000","last_fullsized":"2023-10-30T12:57:39.642976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:34.799606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'326","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574849+0000","last_change":"2023-10-30T12:52:38.168173+0000","last_active":"2023-10-30T12:57:21.574849+0000","last_peered":"2023-10-30T12:57:21.574849+0000","last_clean":"2023-10-30T12:57:21.574849+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:57:21.574849+0000","last_undegraded":"2023-10-30T12:57:21.574849+0000","last_fullsized":"2023-10-30T12:57:21.574849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:17:54.844918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767246,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":326,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'339","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677329+0000","last_change":"2023-10-30T12:52:38.960608+0000","last_active":"2023-10-30T12:57:39.677329+0000","last_peered":"2023-10-30T12:57:39.677329+0000","last_clean":"2023-10-30T12:57:39.677329+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:57:39.677329+0000","last_undegraded":"2023-10-30T12:57:39.677329+0000","last_fullsized":"2023-10-30T12:57:39.677329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:35.999124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'337","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642756+0000","last_change":"2023-10-30T12:52:38.959629+0000","last_active":"2023-10-30T12:57:39.642756+0000","last_peered":"2023-10-30T12:57:39.642756+0000","last_clean":"2023-10-30T12:57:39.642756+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:57:39.642756+0000","last_undegraded":"2023-10-30T12:57:39.642756+0000","last_fullsized":"2023-10-30T12:57:39.642756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:11:54.888888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'312","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674922+0000","last_change":"2023-10-30T12:52:38.378139+0000","last_active":"2023-10-30T12:57:39.674922+0000","last_peered":"2023-10-30T12:57:39.674922+0000","last_clean":"2023-10-30T12:57:39.674922+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:39.674922+0000","last_undegraded":"2023-10-30T12:57:39.674922+0000","last_fullsized":"2023-10-30T12:57:39.674922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:34.995464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'314","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.636430+0000","last_change":"2023-10-30T12:52:38.374275+0000","last_active":"2023-10-30T12:57:39.636430+0000","last_peered":"2023-10-30T12:57:39.636430+0000","last_clean":"2023-10-30T12:57:39.636430+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:57:39.636430+0000","last_undegraded":"2023-10-30T12:57:39.636430+0000","last_fullsized":"2023-10-30T12:57:39.636430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:43:44.654314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'352","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621960+0000","last_change":"2023-10-30T12:52:38.161570+0000","last_active":"2023-10-30T12:57:39.621960+0000","last_peered":"2023-10-30T12:57:39.621960+0000","last_clean":"2023-10-30T12:57:39.621960+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:57:39.621960+0000","last_undegraded":"2023-10-30T12:57:39.621960+0000","last_fullsized":"2023-10-30T12:57:39.621960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:20.253861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'344","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660738+0000","last_change":"2023-10-30T12:52:38.157652+0000","last_active":"2023-10-30T12:57:39.660738+0000","last_peered":"2023-10-30T12:57:39.660738+0000","last_clean":"2023-10-30T12:57:39.660738+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:57:39.660738+0000","last_undegraded":"2023-10-30T12:57:39.660738+0000","last_fullsized":"2023-10-30T12:57:39.660738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:45:33.635954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'358","reported_seq":2481,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561366+0000","last_change":"2023-10-30T12:52:38.190778+0000","last_active":"2023-10-30T12:57:21.561366+0000","last_peered":"2023-10-30T12:57:21.561366+0000","last_clean":"2023-10-30T12:57:21.561366+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:57:21.561366+0000","last_undegraded":"2023-10-30T12:57:21.561366+0000","last_fullsized":"2023-10-30T12:57:21.561366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:23.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23735274,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":356,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.568060+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:21.568060+0000","last_peered":"2023-10-30T12:57:21.568060+0000","last_clean":"2023-10-30T12:57:21.568060+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:21.568060+0000","last_undegraded":"2023-10-30T12:57:21.568060+0000","last_fullsized":"2023-10-30T12:57:21.568060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:39.852314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10012551,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":322,"num_write_kb":14046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'346","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638116+0000","last_change":"2023-10-30T12:52:38.708281+0000","last_active":"2023-10-30T12:57:39.638116+0000","last_peered":"2023-10-30T12:57:39.638116+0000","last_clean":"2023-10-30T12:57:39.638116+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:57:39.638116+0000","last_undegraded":"2023-10-30T12:57:39.638116+0000","last_fullsized":"2023-10-30T12:57:39.638116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:24:56.978901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'341","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573140+0000","last_change":"2023-10-30T12:52:38.166885+0000","last_active":"2023-10-30T12:57:21.573140+0000","last_peered":"2023-10-30T12:57:21.573140+0000","last_clean":"2023-10-30T12:57:21.573140+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:57:21.573140+0000","last_undegraded":"2023-10-30T12:57:21.573140+0000","last_fullsized":"2023-10-30T12:57:21.573140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:17:23.403084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695056,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":340,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'351","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670046+0000","last_change":"2023-10-30T12:52:38.959805+0000","last_active":"2023-10-30T12:57:39.670046+0000","last_peered":"2023-10-30T12:57:39.670046+0000","last_clean":"2023-10-30T12:57:39.670046+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:57:39.670046+0000","last_undegraded":"2023-10-30T12:57:39.670046+0000","last_fullsized":"2023-10-30T12:57:39.670046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:45.750836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'355","reported_seq":2479,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671742+0000","last_change":"2023-10-30T12:52:38.159940+0000","last_active":"2023-10-30T12:57:39.671742+0000","last_peered":"2023-10-30T12:57:39.671742+0000","last_clean":"2023-10-30T12:57:39.671742+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:57:39.671742+0000","last_undegraded":"2023-10-30T12:57:39.671742+0000","last_fullsized":"2023-10-30T12:57:39.671742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:51:28.603483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'383","reported_seq":2506,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.648123+0000","last_change":"2023-10-30T12:52:38.707969+0000","last_active":"2023-10-30T12:57:39.648123+0000","last_peered":"2023-10-30T12:57:39.648123+0000","last_clean":"2023-10-30T12:57:39.648123+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:57:39.648123+0000","last_undegraded":"2023-10-30T12:57:39.648123+0000","last_fullsized":"2023-10-30T12:57:39.648123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:50.085992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'372","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659940+0000","last_change":"2023-10-30T12:52:38.374161+0000","last_active":"2023-10-30T12:57:39.659940+0000","last_peered":"2023-10-30T12:57:39.659940+0000","last_clean":"2023-10-30T12:57:39.659940+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:57:39.659940+0000","last_undegraded":"2023-10-30T12:57:39.659940+0000","last_fullsized":"2023-10-30T12:57:39.659940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:05.818986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'343","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561012+0000","last_change":"2023-10-30T12:52:38.192255+0000","last_active":"2023-10-30T12:57:21.561012+0000","last_peered":"2023-10-30T12:57:21.561012+0000","last_clean":"2023-10-30T12:57:21.561012+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:57:21.561012+0000","last_undegraded":"2023-10-30T12:57:21.561012+0000","last_fullsized":"2023-10-30T12:57:21.561012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:37.588996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'327","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658302+0000","last_change":"2023-10-30T12:52:38.374594+0000","last_active":"2023-10-30T12:57:39.658302+0000","last_peered":"2023-10-30T12:57:39.658302+0000","last_clean":"2023-10-30T12:57:39.658302+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:57:39.658302+0000","last_undegraded":"2023-10-30T12:57:39.658302+0000","last_fullsized":"2023-10-30T12:57:39.658302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:08.496852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'379","reported_seq":2510,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.669068+0000","last_change":"2023-10-30T12:52:38.960525+0000","last_active":"2023-10-30T12:57:39.669068+0000","last_peered":"2023-10-30T12:57:39.669068+0000","last_clean":"2023-10-30T12:57:39.669068+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:57:39.669068+0000","last_undegraded":"2023-10-30T12:57:39.669068+0000","last_fullsized":"2023-10-30T12:57:39.669068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:09.235440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617272+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:39.617272+0000","last_peered":"2023-10-30T12:57:39.617272+0000","last_clean":"2023-10-30T12:57:39.617272+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:39.617272+0000","last_undegraded":"2023-10-30T12:57:39.617272+0000","last_fullsized":"2023-10-30T12:57:39.617272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:38.878517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'306","reported_seq":2419,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670815+0000","last_change":"2023-10-30T12:52:38.375028+0000","last_active":"2023-10-30T12:57:39.670815+0000","last_peered":"2023-10-30T12:57:39.670815+0000","last_clean":"2023-10-30T12:57:39.670815+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:57:39.670815+0000","last_undegraded":"2023-10-30T12:57:39.670815+0000","last_fullsized":"2023-10-30T12:57:39.670815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:53.865022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'330","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.618194+0000","last_change":"2023-10-30T12:52:38.160968+0000","last_active":"2023-10-30T12:57:39.618194+0000","last_peered":"2023-10-30T12:57:39.618194+0000","last_clean":"2023-10-30T12:57:39.618194+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:39.618194+0000","last_undegraded":"2023-10-30T12:57:39.618194+0000","last_fullsized":"2023-10-30T12:57:39.618194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:14:14.802106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'360","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560965+0000","last_change":"2023-10-30T12:52:38.193081+0000","last_active":"2023-10-30T12:57:21.560965+0000","last_peered":"2023-10-30T12:57:21.560965+0000","last_clean":"2023-10-30T12:57:21.560965+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:57:21.560965+0000","last_undegraded":"2023-10-30T12:57:21.560965+0000","last_fullsized":"2023-10-30T12:57:21.560965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:19.937261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15312207,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":357,"num_write_kb":15140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'376","reported_seq":2497,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642772+0000","last_change":"2023-10-30T12:52:38.708132+0000","last_active":"2023-10-30T12:57:39.642772+0000","last_peered":"2023-10-30T12:57:39.642772+0000","last_clean":"2023-10-30T12:57:39.642772+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:57:39.642772+0000","last_undegraded":"2023-10-30T12:57:39.642772+0000","last_fullsized":"2023-10-30T12:57:39.642772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:23:07.870943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'336","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656156+0000","last_change":"2023-10-30T12:52:38.956531+0000","last_active":"2023-10-30T12:57:39.656156+0000","last_peered":"2023-10-30T12:57:39.656156+0000","last_clean":"2023-10-30T12:57:39.656156+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:57:39.656156+0000","last_undegraded":"2023-10-30T12:57:39.656156+0000","last_fullsized":"2023-10-30T12:57:39.656156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:37.995081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'342","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657965+0000","last_change":"2023-10-30T12:52:38.377927+0000","last_active":"2023-10-30T12:57:39.657965+0000","last_peered":"2023-10-30T12:57:39.657965+0000","last_clean":"2023-10-30T12:57:39.657965+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:39.657965+0000","last_undegraded":"2023-10-30T12:57:39.657965+0000","last_fullsized":"2023-10-30T12:57:39.657965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:29.073770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"36'262","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258065+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:57:12.258065+0000","last_peered":"2023-10-30T12:57:12.258065+0000","last_clean":"2023-10-30T12:57:12.258065+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:12.258065+0000","last_undegraded":"2023-10-30T12:57:12.258065+0000","last_fullsized":"2023-10-30T12:57:12.258065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'324","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657884+0000","last_change":"2023-10-30T12:52:38.161345+0000","last_active":"2023-10-30T12:57:39.657884+0000","last_peered":"2023-10-30T12:57:39.657884+0000","last_clean":"2023-10-30T12:57:39.657884+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:39.657884+0000","last_undegraded":"2023-10-30T12:57:39.657884+0000","last_fullsized":"2023-10-30T12:57:39.657884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:18.100051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'281","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.366158+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:57:39.366158+0000","last_peered":"2023-10-30T12:57:39.366158+0000","last_clean":"2023-10-30T12:57:39.366158+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:57:39.366158+0000","last_undegraded":"2023-10-30T12:57:39.366158+0000","last_fullsized":"2023-10-30T12:57:39.366158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403693,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":278,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"52'72","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242321+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:57:12.242321+0000","last_peered":"2023-10-30T12:57:12.242321+0000","last_clean":"2023-10-30T12:57:12.242321+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:12.242321+0000","last_undegraded":"2023-10-30T12:57:12.242321+0000","last_fullsized":"2023-10-30T12:57:12.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'400","reported_seq":2531,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.613041+0000","last_change":"2023-10-30T12:52:38.375331+0000","last_active":"2023-10-30T12:57:39.613041+0000","last_peered":"2023-10-30T12:57:39.613041+0000","last_clean":"2023-10-30T12:57:39.613041+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:39.613041+0000","last_undegraded":"2023-10-30T12:57:39.613041+0000","last_fullsized":"2023-10-30T12:57:39.613041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:29.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"52'341","reported_seq":392,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.262243+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:57:12.262243+0000","last_peered":"2023-10-30T12:57:12.262243+0000","last_clean":"2023-10-30T12:57:12.262243+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:12.262243+0000","last_undegraded":"2023-10-30T12:57:12.262243+0000","last_fullsized":"2023-10-30T12:57:12.262243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'356","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617014+0000","last_change":"2023-10-30T12:52:38.157348+0000","last_active":"2023-10-30T12:57:39.617014+0000","last_peered":"2023-10-30T12:57:39.617014+0000","last_clean":"2023-10-30T12:57:39.617014+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:57:39.617014+0000","last_undegraded":"2023-10-30T12:57:39.617014+0000","last_fullsized":"2023-10-30T12:57:39.617014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:06.624515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'31","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:29.786928+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:57:29.786928+0000","last_peered":"2023-10-30T12:57:29.786928+0000","last_clean":"2023-10-30T12:57:29.786928+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:29.786928+0000","last_undegraded":"2023-10-30T12:57:29.786928+0000","last_fullsized":"2023-10-30T12:57:29.786928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":31,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'350","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571444+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:21.571444+0000","last_peered":"2023-10-30T12:57:21.571444+0000","last_clean":"2023-10-30T12:57:21.571444+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:21.571444+0000","last_undegraded":"2023-10-30T12:57:21.571444+0000","last_fullsized":"2023-10-30T12:57:21.571444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:09.875012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:24.827512+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:57:24.827512+0000","last_peered":"2023-10-30T12:57:24.827512+0000","last_clean":"2023-10-30T12:57:24.827512+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:57:24.827512+0000","last_undegraded":"2023-10-30T12:57:24.827512+0000","last_fullsized":"2023-10-30T12:57:24.827512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069734,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":1048,"num_write":70,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'349","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.622444+0000","last_change":"2023-10-30T12:52:38.162408+0000","last_active":"2023-10-30T12:57:39.622444+0000","last_peered":"2023-10-30T12:57:39.622444+0000","last_clean":"2023-10-30T12:57:39.622444+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:57:39.622444+0000","last_undegraded":"2023-10-30T12:57:39.622444+0000","last_fullsized":"2023-10-30T12:57:39.622444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:14.339468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"52'43","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.246572+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:57:12.246572+0000","last_peered":"2023-10-30T12:57:12.246572+0000","last_clean":"2023-10-30T12:57:12.246572+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:12.246572+0000","last_undegraded":"2023-10-30T12:57:12.246572+0000","last_fullsized":"2023-10-30T12:57:12.246572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'332","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674677+0000","last_change":"2023-10-30T12:52:38.378836+0000","last_active":"2023-10-30T12:57:39.674677+0000","last_peered":"2023-10-30T12:57:39.674677+0000","last_clean":"2023-10-30T12:57:39.674677+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:39.674677+0000","last_undegraded":"2023-10-30T12:57:39.674677+0000","last_fullsized":"2023-10-30T12:57:39.674677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:57:03.390521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'107","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:31.407376+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:57:31.407376+0000","last_peered":"2023-10-30T12:57:31.407376+0000","last_clean":"2023-10-30T12:57:31.407376+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:31.407376+0000","last_undegraded":"2023-10-30T12:57:31.407376+0000","last_fullsized":"2023-10-30T12:57:31.407376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":106,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'306","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573471+0000","last_change":"2023-10-30T12:52:38.163983+0000","last_active":"2023-10-30T12:57:21.573471+0000","last_peered":"2023-10-30T12:57:21.573471+0000","last_clean":"2023-10-30T12:57:21.573471+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:57:21.573471+0000","last_undegraded":"2023-10-30T12:57:21.573471+0000","last_fullsized":"2023-10-30T12:57:21.573471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:26.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"52'354","reported_seq":416,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608375+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:57:07.608375+0000","last_peered":"2023-10-30T12:57:07.608375+0000","last_clean":"2023-10-30T12:57:07.608375+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:07.608375+0000","last_undegraded":"2023-10-30T12:57:07.608375+0000","last_fullsized":"2023-10-30T12:57:07.608375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'359","reported_seq":2485,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.661402+0000","last_change":"2023-10-30T12:52:38.161686+0000","last_active":"2023-10-30T12:57:39.661402+0000","last_peered":"2023-10-30T12:57:39.661402+0000","last_clean":"2023-10-30T12:57:39.661402+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:39.661402+0000","last_undegraded":"2023-10-30T12:57:39.661402+0000","last_fullsized":"2023-10-30T12:57:39.661402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:52.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"52'104","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.611180+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:57:07.611180+0000","last_peered":"2023-10-30T12:57:07.611180+0000","last_clean":"2023-10-30T12:57:07.611180+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:07.611180+0000","last_undegraded":"2023-10-30T12:57:07.611180+0000","last_fullsized":"2023-10-30T12:57:07.611180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":7385,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'345","reported_seq":2482,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571576+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:21.571576+0000","last_peered":"2023-10-30T12:57:21.571576+0000","last_clean":"2023-10-30T12:57:21.571576+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:21.571576+0000","last_undegraded":"2023-10-30T12:57:21.571576+0000","last_fullsized":"2023-10-30T12:57:21.571576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:51.506289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'394","reported_seq":453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.333732+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:57:39.333732+0000","last_peered":"2023-10-30T12:57:39.333732+0000","last_clean":"2023-10-30T12:57:39.333732+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:57:39.333732+0000","last_undegraded":"2023-10-30T12:57:39.333732+0000","last_fullsized":"2023-10-30T12:57:39.333732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16248710,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":387,"num_write_kb":16075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'330","reported_seq":2447,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656223+0000","last_change":"2023-10-30T12:52:38.959038+0000","last_active":"2023-10-30T12:57:39.656223+0000","last_peered":"2023-10-30T12:57:39.656223+0000","last_clean":"2023-10-30T12:57:39.656223+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:57:39.656223+0000","last_undegraded":"2023-10-30T12:57:39.656223+0000","last_fullsized":"2023-10-30T12:57:39.656223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:03:23.132047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"52'112","reported_seq":181,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608639+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:57:07.608639+0000","last_peered":"2023-10-30T12:57:07.608639+0000","last_clean":"2023-10-30T12:57:07.608639+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:07.608639+0000","last_undegraded":"2023-10-30T12:57:07.608639+0000","last_fullsized":"2023-10-30T12:57:07.608639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'357","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.739802+0000","last_change":"2023-10-30T12:52:38.707521+0000","last_active":"2023-10-30T12:57:39.739802+0000","last_peered":"2023-10-30T12:57:39.739802+0000","last_clean":"2023-10-30T12:57:39.739802+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:57:39.739802+0000","last_undegraded":"2023-10-30T12:57:39.739802+0000","last_fullsized":"2023-10-30T12:57:39.739802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:24.903914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'102","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:14.887635+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:57:14.887635+0000","last_peered":"2023-10-30T12:57:14.887635+0000","last_clean":"2023-10-30T12:57:14.887635+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:14.887635+0000","last_undegraded":"2023-10-30T12:57:14.887635+0000","last_fullsized":"2023-10-30T12:57:14.887635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":176,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'304","reported_seq":2424,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573853+0000","last_change":"2023-10-30T12:52:38.166069+0000","last_active":"2023-10-30T12:57:21.573853+0000","last_peered":"2023-10-30T12:57:21.573853+0000","last_clean":"2023-10-30T12:57:21.573853+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:57:21.573853+0000","last_undegraded":"2023-10-30T12:57:21.573853+0000","last_fullsized":"2023-10-30T12:57:21.573853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:36.932929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12071184,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":306,"num_write_kb":15946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"52'165","reported_seq":217,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745677+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:56:34.745677+0000","last_peered":"2023-10-30T12:56:34.745677+0000","last_clean":"2023-10-30T12:56:34.745677+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:56:34.745677+0000","last_undegraded":"2023-10-30T12:56:34.745677+0000","last_fullsized":"2023-10-30T12:56:34.745677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'319","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574457+0000","last_change":"2023-10-30T12:52:38.167701+0000","last_active":"2023-10-30T12:57:21.574457+0000","last_peered":"2023-10-30T12:57:21.574457+0000","last_clean":"2023-10-30T12:57:21.574457+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:57:21.574457+0000","last_undegraded":"2023-10-30T12:57:21.574457+0000","last_fullsized":"2023-10-30T12:57:21.574457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:54.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"52'123","reported_seq":174,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.787653+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:56:34.787653+0000","last_peered":"2023-10-30T12:56:34.787653+0000","last_clean":"2023-10-30T12:56:34.787653+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:56:34.787653+0000","last_undegraded":"2023-10-30T12:56:34.787653+0000","last_fullsized":"2023-10-30T12:56:34.787653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'357","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558014+0000","last_change":"2023-10-30T12:52:38.194662+0000","last_active":"2023-10-30T12:57:21.558014+0000","last_peered":"2023-10-30T12:57:21.558014+0000","last_clean":"2023-10-30T12:57:21.558014+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:57:21.558014+0000","last_undegraded":"2023-10-30T12:57:21.558014+0000","last_fullsized":"2023-10-30T12:57:21.558014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:33:24.813894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526551,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":357,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"52'12","reported_seq":70,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745646+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:56:34.745646+0000","last_peered":"2023-10-30T12:56:34.745646+0000","last_clean":"2023-10-30T12:56:34.745646+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:56:34.745646+0000","last_undegraded":"2023-10-30T12:56:34.745646+0000","last_fullsized":"2023-10-30T12:56:34.745646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.570633+0000","last_change":"2023-10-30T12:52:38.166263+0000","last_active":"2023-10-30T12:57:21.570633+0000","last_peered":"2023-10-30T12:57:21.570633+0000","last_clean":"2023-10-30T12:57:21.570633+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:57:21.570633+0000","last_undegraded":"2023-10-30T12:57:21.570633+0000","last_fullsized":"2023-10-30T12:57:21.570633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:44.813227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468332,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":322,"num_write_kb":15461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.243330+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:57:07.243330+0000","last_peered":"2023-10-30T12:57:07.243330+0000","last_clean":"2023-10-30T12:57:07.243330+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:07.243330+0000","last_undegraded":"2023-10-30T12:57:07.243330+0000","last_fullsized":"2023-10-30T12:57:07.243330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'10","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242909+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:57:12.242909+0000","last_peered":"2023-10-30T12:57:12.242909+0000","last_clean":"2023-10-30T12:57:12.242909+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:12.242909+0000","last_undegraded":"2023-10-30T12:57:12.242909+0000","last_fullsized":"2023-10-30T12:57:12.242909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'310","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.608876+0000","last_change":"2023-10-30T12:52:38.374620+0000","last_active":"2023-10-30T12:57:39.608876+0000","last_peered":"2023-10-30T12:57:39.608876+0000","last_clean":"2023-10-30T12:57:39.608876+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:57:39.608876+0000","last_undegraded":"2023-10-30T12:57:39.608876+0000","last_fullsized":"2023-10-30T12:57:39.608876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:36:22.529147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"52'497","reported_seq":557,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.260199+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:57:12.260199+0000","last_peered":"2023-10-30T12:57:12.260199+0000","last_clean":"2023-10-30T12:57:12.260199+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:12.260199+0000","last_undegraded":"2023-10-30T12:57:12.260199+0000","last_fullsized":"2023-10-30T12:57:12.260199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'327","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.663068+0000","last_change":"2023-10-30T12:52:38.159894+0000","last_active":"2023-10-30T12:57:39.663068+0000","last_peered":"2023-10-30T12:57:39.663068+0000","last_clean":"2023-10-30T12:57:39.663068+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:57:39.663068+0000","last_undegraded":"2023-10-30T12:57:39.663068+0000","last_fullsized":"2023-10-30T12:57:39.663068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:01.190957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'139","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258850+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:57:12.258850+0000","last_peered":"2023-10-30T12:57:12.258850+0000","last_clean":"2023-10-30T12:57:12.258850+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:57:12.258850+0000","last_undegraded":"2023-10-30T12:57:12.258850+0000","last_fullsized":"2023-10-30T12:57:12.258850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4302161,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":135,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'325","reported_seq":2452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617476+0000","last_change":"2023-10-30T12:52:38.157754+0000","last_active":"2023-10-30T12:57:39.617476+0000","last_peered":"2023-10-30T12:57:39.617476+0000","last_clean":"2023-10-30T12:57:39.617476+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:57:39.617476+0000","last_undegraded":"2023-10-30T12:57:39.617476+0000","last_fullsized":"2023-10-30T12:57:39.617476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:15:05.869155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"52'245","reported_seq":297,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.453118+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:56:35.453118+0000","last_peered":"2023-10-30T12:56:35.453118+0000","last_clean":"2023-10-30T12:56:35.453118+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:56:35.453118+0000","last_undegraded":"2023-10-30T12:56:35.453118+0000","last_fullsized":"2023-10-30T12:56:35.453118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'373","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.654586+0000","last_change":"2023-10-30T12:52:38.380527+0000","last_active":"2023-10-30T12:57:39.654586+0000","last_peered":"2023-10-30T12:57:39.654586+0000","last_clean":"2023-10-30T12:57:39.654586+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:39.654586+0000","last_undegraded":"2023-10-30T12:57:39.654586+0000","last_fullsized":"2023-10-30T12:57:39.654586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:28:52.581209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'3","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.239512+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:57:07.239512+0000","last_peered":"2023-10-30T12:57:07.239512+0000","last_clean":"2023-10-30T12:57:07.239512+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:57:07.239512+0000","last_undegraded":"2023-10-30T12:57:07.239512+0000","last_fullsized":"2023-10-30T12:57:07.239512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'315","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558514+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:21.558514+0000","last_peered":"2023-10-30T12:57:21.558514+0000","last_clean":"2023-10-30T12:57:21.558514+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:21.558514+0000","last_undegraded":"2023-10-30T12:57:21.558514+0000","last_fullsized":"2023-10-30T12:57:21.558514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:35.262822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9696989,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":311,"num_write_kb":13726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'9","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.243063+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:57:12.243063+0000","last_peered":"2023-10-30T12:57:12.243063+0000","last_clean":"2023-10-30T12:57:12.243063+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:12.243063+0000","last_undegraded":"2023-10-30T12:57:12.243063+0000","last_fullsized":"2023-10-30T12:57:12.243063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'341","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617288+0000","last_change":"2023-10-30T12:52:38.158164+0000","last_active":"2023-10-30T12:57:39.617288+0000","last_peered":"2023-10-30T12:57:39.617288+0000","last_clean":"2023-10-30T12:57:39.617288+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:57:39.617288+0000","last_undegraded":"2023-10-30T12:57:39.617288+0000","last_fullsized":"2023-10-30T12:57:39.617288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:42.652706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'99","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:26.427271+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:57:26.427271+0000","last_peered":"2023-10-30T12:57:26.427271+0000","last_clean":"2023-10-30T12:57:26.427271+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:57:26.427271+0000","last_undegraded":"2023-10-30T12:57:26.427271+0000","last_fullsized":"2023-10-30T12:57:26.427271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":238,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'314","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.655926+0000","last_change":"2023-10-30T12:52:38.374909+0000","last_active":"2023-10-30T12:57:39.655926+0000","last_peered":"2023-10-30T12:57:39.655926+0000","last_clean":"2023-10-30T12:57:39.655926+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:57:39.655926+0000","last_undegraded":"2023-10-30T12:57:39.655926+0000","last_fullsized":"2023-10-30T12:57:39.655926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:27:11.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"52'12","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610156+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:57:07.610156+0000","last_peered":"2023-10-30T12:57:07.610156+0000","last_clean":"2023-10-30T12:57:07.610156+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:07.610156+0000","last_undegraded":"2023-10-30T12:57:07.610156+0000","last_fullsized":"2023-10-30T12:57:07.610156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'331","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617128+0000","last_change":"2023-10-30T12:52:38.161992+0000","last_active":"2023-10-30T12:57:39.617128+0000","last_peered":"2023-10-30T12:57:39.617128+0000","last_clean":"2023-10-30T12:57:39.617128+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:39.617128+0000","last_undegraded":"2023-10-30T12:57:39.617128+0000","last_fullsized":"2023-10-30T12:57:39.617128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:36.903572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'163","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.279523+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:57:12.279523+0000","last_peered":"2023-10-30T12:57:12.279523+0000","last_clean":"2023-10-30T12:57:12.279523+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:12.279523+0000","last_undegraded":"2023-10-30T12:57:12.279523+0000","last_fullsized":"2023-10-30T12:57:12.279523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'340","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668698+0000","last_change":"2023-10-30T12:52:38.380106+0000","last_active":"2023-10-30T12:57:39.668698+0000","last_peered":"2023-10-30T12:57:39.668698+0000","last_clean":"2023-10-30T12:57:39.668698+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:39.668698+0000","last_undegraded":"2023-10-30T12:57:39.668698+0000","last_fullsized":"2023-10-30T12:57:39.668698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:12:33.391336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"52'58","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.324062+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:57:35.324062+0000","last_peered":"2023-10-30T12:57:35.324062+0000","last_clean":"2023-10-30T12:57:35.324062+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:57:35.324062+0000","last_undegraded":"2023-10-30T12:57:35.324062+0000","last_fullsized":"2023-10-30T12:57:35.324062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'327","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.579159+0000","last_change":"2023-10-30T12:52:38.707846+0000","last_active":"2023-10-30T12:57:39.579159+0000","last_peered":"2023-10-30T12:57:39.579159+0000","last_clean":"2023-10-30T12:57:39.579159+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:57:39.579159+0000","last_undegraded":"2023-10-30T12:57:39.579159+0000","last_fullsized":"2023-10-30T12:57:39.579159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:38:49.638944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'118","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.416434+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:57:19.416434+0000","last_peered":"2023-10-30T12:57:19.416434+0000","last_clean":"2023-10-30T12:57:19.416434+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:57:19.416434+0000","last_undegraded":"2023-10-30T12:57:19.416434+0000","last_fullsized":"2023-10-30T12:57:19.416434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":197,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'330","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602487+0000","last_change":"2023-10-30T12:52:38.373796+0000","last_active":"2023-10-30T12:57:39.602487+0000","last_peered":"2023-10-30T12:57:39.602487+0000","last_clean":"2023-10-30T12:57:39.602487+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:57:39.602487+0000","last_undegraded":"2023-10-30T12:57:39.602487+0000","last_fullsized":"2023-10-30T12:57:39.602487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:16.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"52'187","reported_seq":241,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.833028+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:56:34.833028+0000","last_peered":"2023-10-30T12:56:34.833028+0000","last_clean":"2023-10-30T12:56:34.833028+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:56:34.833028+0000","last_undegraded":"2023-10-30T12:56:34.833028+0000","last_fullsized":"2023-10-30T12:56:34.833028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":4104,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'332","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659101+0000","last_change":"2023-10-30T12:52:38.955966+0000","last_active":"2023-10-30T12:57:39.659101+0000","last_peered":"2023-10-30T12:57:39.659101+0000","last_clean":"2023-10-30T12:57:39.659101+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:57:39.659101+0000","last_undegraded":"2023-10-30T12:57:39.659101+0000","last_fullsized":"2023-10-30T12:57:39.659101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:55.980109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'314","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558152+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:21.558152+0000","last_peered":"2023-10-30T12:57:21.558152+0000","last_clean":"2023-10-30T12:57:21.558152+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:21.558152+0000","last_undegraded":"2023-10-30T12:57:21.558152+0000","last_fullsized":"2023-10-30T12:57:21.558152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:52.978943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22515664,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":311,"num_write_kb":26246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"52'151","reported_seq":202,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.983897+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:56:29.983897+0000","last_peered":"2023-10-30T12:56:29.983897+0000","last_clean":"2023-10-30T12:56:29.983897+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:56:29.983897+0000","last_undegraded":"2023-10-30T12:56:29.983897+0000","last_fullsized":"2023-10-30T12:56:29.983897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'315","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560674+0000","last_change":"2023-10-30T12:52:38.194210+0000","last_active":"2023-10-30T12:57:21.560674+0000","last_peered":"2023-10-30T12:57:21.560674+0000","last_clean":"2023-10-30T12:57:21.560674+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:57:21.560674+0000","last_undegraded":"2023-10-30T12:57:21.560674+0000","last_fullsized":"2023-10-30T12:57:21.560674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:56:10.108457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242644,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":313,"num_write_kb":14254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"52'362","reported_seq":420,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.158457+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:56:40.158457+0000","last_peered":"2023-10-30T12:56:40.158457+0000","last_clean":"2023-10-30T12:56:40.158457+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:56:40.158457+0000","last_undegraded":"2023-10-30T12:56:40.158457+0000","last_fullsized":"2023-10-30T12:56:40.158457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642959+0000","last_change":"2023-10-30T12:52:38.162371+0000","last_active":"2023-10-30T12:57:39.642959+0000","last_peered":"2023-10-30T12:57:39.642959+0000","last_clean":"2023-10-30T12:57:39.642959+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:57:39.642959+0000","last_undegraded":"2023-10-30T12:57:39.642959+0000","last_fullsized":"2023-10-30T12:57:39.642959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:05:58.391344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"52'278","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610628+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:57:07.610628+0000","last_peered":"2023-10-30T12:57:07.610628+0000","last_clean":"2023-10-30T12:57:07.610628+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:07.610628+0000","last_undegraded":"2023-10-30T12:57:07.610628+0000","last_fullsized":"2023-10-30T12:57:07.610628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'331","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638149+0000","last_change":"2023-10-30T12:52:38.373954+0000","last_active":"2023-10-30T12:57:39.638149+0000","last_peered":"2023-10-30T12:57:39.638149+0000","last_clean":"2023-10-30T12:57:39.638149+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:57:39.638149+0000","last_undegraded":"2023-10-30T12:57:39.638149+0000","last_fullsized":"2023-10-30T12:57:39.638149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:21.106315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"52'193","reported_seq":257,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.204972+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:56:14.204972+0000","last_peered":"2023-10-30T12:56:14.204972+0000","last_clean":"2023-10-30T12:56:14.204972+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:56:14.204972+0000","last_undegraded":"2023-10-30T12:56:14.204972+0000","last_fullsized":"2023-10-30T12:56:14.204972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925148655,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20870,"num_write_kb":1047645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1900888064,"data_stored":1847301778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21588,"ondisk_log_size":21588,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":174815753,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":580,"num_read_kb":143240,"num_write":5366,"num_write_kb":176360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350330880,"data_stored":350174933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5098,"ondisk_log_size":5098,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405832,"kb_used_data":326080,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93966008,"statfs":{"total":96636764160,"available":96221192192,"internally_reserved":0,"allocated":333905920,"data_stored":326089857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":81656267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.01}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":0.78400000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":1.272},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.073}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.399}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.327},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":1.4870000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281376,"kb_used_data":236312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090464,"statfs":{"total":96636764160,"available":96348635136,"internally_reserved":0,"allocated":241983488,"data_stored":236763651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.624}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.748}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.1319999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359668,"kb_used_data":292396,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94012172,"statfs":{"total":96636764160,"available":96268464128,"internally_reserved":0,"allocated":299413504,"data_stored":291703362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":1.046}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":1.121}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":1.147}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.008}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.75}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.2689999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416500,"kb_used_data":340908,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93955340,"statfs":{"total":96636764160,"available":96210268160,"internally_reserved":0,"allocated":349089792,"data_stored":340928408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":0.84299999999999997}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.753},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89700000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":0.999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.0529999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406132,"kb_used_data":334700,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93965708,"statfs":{"total":96636764160,"available":96220884992,"internally_reserved":0,"allocated":342732800,"data_stored":334429120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73136584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.60599999999999998,"5min":0.54100000000000004,"15min":0.499},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.786,"5min":1.867,"15min":1.867},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":176240,"kb_used_data":149224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94195600,"statfs":{"total":96636764160,"available":96456294400,"internally_reserved":0,"allocated":152805376,"data_stored":147769036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.496},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.68600000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266192,"kb_used_data":212808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94105648,"statfs":{"total":96636764160,"available":96364183552,"internally_reserved":0,"allocated":217915392,"data_stored":212359979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.92900000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":0.85199999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381824,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990016,"statfs":{"total":96636764160,"available":96245776384,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.374},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.45200000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12713984,"data_stored":12691497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42254336,"data_stored":42233706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":60223488,"data_stored":60206961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":34648064,"data_stored":34625544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49438720,"data_stored":49419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264155136,"data_stored":256775105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":261382144,"data_stored":253892880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:40.777 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T12:57:40.777 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:40.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 Processor -- start 2023-10-30T12:57:40.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 -- start start 2023-10-30T12:57:40.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcc241537f0 0x7fcc240701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc2406b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc240c2aa0 con 0x7fcc2415a2e0 2023-10-30T12:57:40.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240c3ab0 con 0x7fcc241537f0 2023-10-30T12:57:40.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2bcf7640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240706e0 con 0x7fcc2406b2f0 2023-10-30T12:57:40.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc29a93640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcc241537f0 0x7fcc240701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc29292640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fcc2415a2e0 0x7fcc240ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37958/0 (socket says 172.21.15.8:37958) 2023-10-30T12:57:40.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc29292640 1 -- 172.21.15.8:0/4194578163 learned_addr learned my addr 172.21.15.8:0/4194578163 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:40.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc29a93640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fcc241537f0 0x7fcc240701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48108/0 (socket says 172.21.15.8:48108) 2023-10-30T12:57:40.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.954+0000 7fcc2a294640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc2406b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc29a93640 1 -- 172.21.15.8:0/4194578163 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 msgr2=0x7fcc2406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:40.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc29a93640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc2406b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc29a93640 1 -- 172.21.15.8:0/4194578163 >> v1:172.21.15.8:6789/0 conn(0x7fcc2415a2e0 legacy=0x7fcc240ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc29a93640 1 -- 172.21.15.8:0/4194578163 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2406bc80 con 0x7fcc241537f0 2023-10-30T12:57:40.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2a294640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc2406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:40.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc28a91640 1 -- 172.21.15.8:0/4194578163 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486851208 0 0) 0x7fcc240c2aa0 con 0x7fcc2415a2e0 2023-10-30T12:57:40.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc29a93640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240701a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097c0 tx=0x7fcc2001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710a664cff83f1b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc28a91640 1 -- 172.21.15.8:0/4194578163 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200a8970 con 0x7fcc241537f0 2023-10-30T12:57:40.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc28a91640 1 -- 172.21.15.8:0/4194578163 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc200a8ad0 con 0x7fcc241537f0 2023-10-30T12:57:40.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc28a91640 1 -- 172.21.15.8:0/4194578163 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200a8e60 con 0x7fcc241537f0 2023-10-30T12:57:40.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/4194578163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 msgr2=0x7fcc240701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240701a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097c0 tx=0x7fcc2001ec70 comp rx=0 tx=0).stop 2023-10-30T12:57:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/4194578163 shutdown_connections 2023-10-30T12:57:40.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc2406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/4194578163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240701a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.958+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/4194578163 >> 172.21.15.8:0/4194578163 conn(0x7fcc24071fb0 msgr2=0x7fcc240723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/4194578163 shutdown_connections 2023-10-30T12:57:40.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/4194578163 wait complete. 2023-10-30T12:57:40.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 Processor -- start 2023-10-30T12:57:40.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- start start 2023-10-30T12:57:40.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc24159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc2415a2e0 0x7fcc240ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c5800 con 0x7fcc241537f0 2023-10-30T12:57:40.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc24154d60 con 0x7fcc2415a2e0 2023-10-30T12:57:40.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2bcf7640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241f2240 con 0x7fcc2406b2f0 2023-10-30T12:57:40.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48120/0 (socket says 172.21.15.8:48120) 2023-10-30T12:57:40.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 -- 172.21.15.8:0/1045063995 learned_addr learned my addr 172.21.15.8:0/1045063995 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:40.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29a93640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc24159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc2a294640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc2415a2e0 0x7fcc240ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 -- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 msgr2=0x7fcc24159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc24159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 -- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc2415a2e0 msgr2=0x7fcc240ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:40.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc2415a2e0 0x7fcc240ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:40.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 -- 172.21.15.8:0/1045063995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20009420 con 0x7fcc241537f0 2023-10-30T12:57:40.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29a93640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc24159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:40.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc29292640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc140011f0 tx=0x7fcc1400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.962+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140074e0 con 0x7fcc241537f0 2023-10-30T12:57:40.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14007640 con 0x7fcc241537f0 2023-10-30T12:57:40.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14004a10 con 0x7fcc241537f0 2023-10-30T12:57:40.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc241f2dc0 con 0x7fcc241537f0 2023-10-30T12:57:40.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc241f2490 con 0x7fcc241537f0 2023-10-30T12:57:40.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2404e170 con 0x7fcc241537f0 2023-10-30T12:57:40.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcc14004b70 con 0x7fcc241537f0 2023-10-30T12:57:40.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc1a7fc640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 0x7fcc08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:40.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fcc14083420 con 0x7fcc241537f0 2023-10-30T12:57:40.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.966+0000 7fcc29a93640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 0x7fcc08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:40.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.970+0000 7fcc29a93640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 0x7fcc08068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc200b2000 tx=0x7fcc20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:40.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:40.970+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcc14050600 con 0x7fcc241537f0 2023-10-30T12:57:41.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.126+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc240708e0 con 0x7fcc080664a0 2023-10-30T12:57:41.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc1a7fc640 1 -- 172.21.15.8:0/1045063995 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292338 (secure 0 0 0) 0x7fcc200b0540 con 0x7fcc080664a0 2023-10-30T12:57:41.141 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:41.142 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:57:41.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 msgr2=0x7fcc08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 0x7fcc08068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc200b2000 tx=0x7fcc20008040 comp rx=0 tx=0).stop 2023-10-30T12:57:41.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 msgr2=0x7fcc240c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc140011f0 tx=0x7fcc1400c9b0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 shutdown_connections 2023-10-30T12:57:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fcc080664a0 0x7fcc08068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc2415a2e0 0x7fcc240ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc241537f0 0x7fcc240c9bd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.134+0000 7fcc2bcf7640 1 --2- 172.21.15.8:0/1045063995 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fcc2406b2f0 0x7fcc24159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.138+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 >> 172.21.15.8:0/1045063995 conn(0x7fcc24071fb0 msgr2=0x7fcc24152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:41.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.138+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 shutdown_connections 2023-10-30T12:57:41.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.138+0000 7fcc2bcf7640 1 -- 172.21.15.8:0/1045063995 wait complete. 2023-10-30T12:57:41.161 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2023-10-30T12:57:39.807154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1100554776,"num_objects":10461,"num_object_clones":0,"num_object_copies":20922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10461,"num_whiteouts":0,"num_read":2527,"num_read_kb":243523,"num_write":26395,"num_write_kb":1226421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26803,"ondisk_log_size":26803,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2693764,"kb_used_data":2202820,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":752280956,"statfs":{"total":773094113280,"available":770335698944,"internally_reserved":0,"allocated":2255687680,"data_stored":2199551815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":502648378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2581772,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":178,"num_write_kb":2634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"3.3f","version":"53'324","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574594+0000","last_change":"2023-10-30T12:52:38.167461+0000","last_active":"2023-10-30T12:57:21.574594+0000","last_peered":"2023-10-30T12:57:21.574594+0000","last_clean":"2023-10-30T12:57:21.574594+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:57:21.574594+0000","last_undegraded":"2023-10-30T12:57:21.574594+0000","last_fullsized":"2023-10-30T12:57:21.574594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:23.473692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'336","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602281+0000","last_change":"2023-10-30T12:52:38.158597+0000","last_active":"2023-10-30T12:57:39.602281+0000","last_peered":"2023-10-30T12:57:39.602281+0000","last_clean":"2023-10-30T12:57:39.602281+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:57:39.602281+0000","last_undegraded":"2023-10-30T12:57:39.602281+0000","last_fullsized":"2023-10-30T12:57:39.602281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:51:39.552812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'359","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617323+0000","last_change":"2023-10-30T12:52:38.161971+0000","last_active":"2023-10-30T12:57:39.617323+0000","last_peered":"2023-10-30T12:57:39.617323+0000","last_clean":"2023-10-30T12:57:39.617323+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:39.617323+0000","last_undegraded":"2023-10-30T12:57:39.617323+0000","last_fullsized":"2023-10-30T12:57:39.617323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:05.398156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'289","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.826281+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:19.826281+0000","last_peered":"2023-10-30T12:57:19.826281+0000","last_clean":"2023-10-30T12:57:19.826281+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:19.826281+0000","last_undegraded":"2023-10-30T12:57:19.826281+0000","last_fullsized":"2023-10-30T12:57:19.826281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:39.214543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571441+0000","last_change":"2023-10-30T12:52:38.166749+0000","last_active":"2023-10-30T12:57:21.571441+0000","last_peered":"2023-10-30T12:57:21.571441+0000","last_clean":"2023-10-30T12:57:21.571441+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:57:21.571441+0000","last_undegraded":"2023-10-30T12:57:21.571441+0000","last_fullsized":"2023-10-30T12:57:21.571441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:27.480500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28212871,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":331,"num_write_kb":27730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'323","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561161+0000","last_change":"2023-10-30T12:52:38.190432+0000","last_active":"2023-10-30T12:57:21.561161+0000","last_peered":"2023-10-30T12:57:21.561161+0000","last_clean":"2023-10-30T12:57:21.561161+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:57:21.561161+0000","last_undegraded":"2023-10-30T12:57:21.561161+0000","last_fullsized":"2023-10-30T12:57:21.561161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:12.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.562949+0000","last_change":"2023-10-30T12:52:38.164109+0000","last_active":"2023-10-30T12:57:21.562949+0000","last_peered":"2023-10-30T12:57:21.562949+0000","last_clean":"2023-10-30T12:57:21.562949+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:21.562949+0000","last_undegraded":"2023-10-30T12:57:21.562949+0000","last_fullsized":"2023-10-30T12:57:21.562949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:03:58.478040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23818883,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":355,"num_write_kb":23453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'316","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642976+0000","last_change":"2023-10-30T12:52:38.375236+0000","last_active":"2023-10-30T12:57:39.642976+0000","last_peered":"2023-10-30T12:57:39.642976+0000","last_clean":"2023-10-30T12:57:39.642976+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:57:39.642976+0000","last_undegraded":"2023-10-30T12:57:39.642976+0000","last_fullsized":"2023-10-30T12:57:39.642976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:34.799606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'326","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574849+0000","last_change":"2023-10-30T12:52:38.168173+0000","last_active":"2023-10-30T12:57:21.574849+0000","last_peered":"2023-10-30T12:57:21.574849+0000","last_clean":"2023-10-30T12:57:21.574849+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:57:21.574849+0000","last_undegraded":"2023-10-30T12:57:21.574849+0000","last_fullsized":"2023-10-30T12:57:21.574849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:17:54.844918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767246,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":326,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'339","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677329+0000","last_change":"2023-10-30T12:52:38.960608+0000","last_active":"2023-10-30T12:57:39.677329+0000","last_peered":"2023-10-30T12:57:39.677329+0000","last_clean":"2023-10-30T12:57:39.677329+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:57:39.677329+0000","last_undegraded":"2023-10-30T12:57:39.677329+0000","last_fullsized":"2023-10-30T12:57:39.677329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:35.999124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'337","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642756+0000","last_change":"2023-10-30T12:52:38.959629+0000","last_active":"2023-10-30T12:57:39.642756+0000","last_peered":"2023-10-30T12:57:39.642756+0000","last_clean":"2023-10-30T12:57:39.642756+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:57:39.642756+0000","last_undegraded":"2023-10-30T12:57:39.642756+0000","last_fullsized":"2023-10-30T12:57:39.642756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:11:54.888888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'312","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674922+0000","last_change":"2023-10-30T12:52:38.378139+0000","last_active":"2023-10-30T12:57:39.674922+0000","last_peered":"2023-10-30T12:57:39.674922+0000","last_clean":"2023-10-30T12:57:39.674922+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:39.674922+0000","last_undegraded":"2023-10-30T12:57:39.674922+0000","last_fullsized":"2023-10-30T12:57:39.674922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:34.995464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'314","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.636430+0000","last_change":"2023-10-30T12:52:38.374275+0000","last_active":"2023-10-30T12:57:39.636430+0000","last_peered":"2023-10-30T12:57:39.636430+0000","last_clean":"2023-10-30T12:57:39.636430+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:57:39.636430+0000","last_undegraded":"2023-10-30T12:57:39.636430+0000","last_fullsized":"2023-10-30T12:57:39.636430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:43:44.654314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'352","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621960+0000","last_change":"2023-10-30T12:52:38.161570+0000","last_active":"2023-10-30T12:57:39.621960+0000","last_peered":"2023-10-30T12:57:39.621960+0000","last_clean":"2023-10-30T12:57:39.621960+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:57:39.621960+0000","last_undegraded":"2023-10-30T12:57:39.621960+0000","last_fullsized":"2023-10-30T12:57:39.621960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:20.253861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'344","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660738+0000","last_change":"2023-10-30T12:52:38.157652+0000","last_active":"2023-10-30T12:57:39.660738+0000","last_peered":"2023-10-30T12:57:39.660738+0000","last_clean":"2023-10-30T12:57:39.660738+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:57:39.660738+0000","last_undegraded":"2023-10-30T12:57:39.660738+0000","last_fullsized":"2023-10-30T12:57:39.660738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:45:33.635954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'358","reported_seq":2481,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561366+0000","last_change":"2023-10-30T12:52:38.190778+0000","last_active":"2023-10-30T12:57:21.561366+0000","last_peered":"2023-10-30T12:57:21.561366+0000","last_clean":"2023-10-30T12:57:21.561366+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:57:21.561366+0000","last_undegraded":"2023-10-30T12:57:21.561366+0000","last_fullsized":"2023-10-30T12:57:21.561366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:23.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23735274,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":356,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.568060+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:21.568060+0000","last_peered":"2023-10-30T12:57:21.568060+0000","last_clean":"2023-10-30T12:57:21.568060+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:21.568060+0000","last_undegraded":"2023-10-30T12:57:21.568060+0000","last_fullsized":"2023-10-30T12:57:21.568060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:39.852314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10012551,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":322,"num_write_kb":14046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'346","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638116+0000","last_change":"2023-10-30T12:52:38.708281+0000","last_active":"2023-10-30T12:57:39.638116+0000","last_peered":"2023-10-30T12:57:39.638116+0000","last_clean":"2023-10-30T12:57:39.638116+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:57:39.638116+0000","last_undegraded":"2023-10-30T12:57:39.638116+0000","last_fullsized":"2023-10-30T12:57:39.638116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:24:56.978901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'341","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573140+0000","last_change":"2023-10-30T12:52:38.166885+0000","last_active":"2023-10-30T12:57:21.573140+0000","last_peered":"2023-10-30T12:57:21.573140+0000","last_clean":"2023-10-30T12:57:21.573140+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:57:21.573140+0000","last_undegraded":"2023-10-30T12:57:21.573140+0000","last_fullsized":"2023-10-30T12:57:21.573140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:17:23.403084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695056,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":340,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'351","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670046+0000","last_change":"2023-10-30T12:52:38.959805+0000","last_active":"2023-10-30T12:57:39.670046+0000","last_peered":"2023-10-30T12:57:39.670046+0000","last_clean":"2023-10-30T12:57:39.670046+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:57:39.670046+0000","last_undegraded":"2023-10-30T12:57:39.670046+0000","last_fullsized":"2023-10-30T12:57:39.670046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:45.750836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'355","reported_seq":2479,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671742+0000","last_change":"2023-10-30T12:52:38.159940+0000","last_active":"2023-10-30T12:57:39.671742+0000","last_peered":"2023-10-30T12:57:39.671742+0000","last_clean":"2023-10-30T12:57:39.671742+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:57:39.671742+0000","last_undegraded":"2023-10-30T12:57:39.671742+0000","last_fullsized":"2023-10-30T12:57:39.671742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:51:28.603483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'383","reported_seq":2506,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.648123+0000","last_change":"2023-10-30T12:52:38.707969+0000","last_active":"2023-10-30T12:57:39.648123+0000","last_peered":"2023-10-30T12:57:39.648123+0000","last_clean":"2023-10-30T12:57:39.648123+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:57:39.648123+0000","last_undegraded":"2023-10-30T12:57:39.648123+0000","last_fullsized":"2023-10-30T12:57:39.648123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:28:50.085992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'372","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659940+0000","last_change":"2023-10-30T12:52:38.374161+0000","last_active":"2023-10-30T12:57:39.659940+0000","last_peered":"2023-10-30T12:57:39.659940+0000","last_clean":"2023-10-30T12:57:39.659940+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:57:39.659940+0000","last_undegraded":"2023-10-30T12:57:39.659940+0000","last_fullsized":"2023-10-30T12:57:39.659940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:05.818986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'343","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.561012+0000","last_change":"2023-10-30T12:52:38.192255+0000","last_active":"2023-10-30T12:57:21.561012+0000","last_peered":"2023-10-30T12:57:21.561012+0000","last_clean":"2023-10-30T12:57:21.561012+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:57:21.561012+0000","last_undegraded":"2023-10-30T12:57:21.561012+0000","last_fullsized":"2023-10-30T12:57:21.561012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:37.588996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'327","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658302+0000","last_change":"2023-10-30T12:52:38.374594+0000","last_active":"2023-10-30T12:57:39.658302+0000","last_peered":"2023-10-30T12:57:39.658302+0000","last_clean":"2023-10-30T12:57:39.658302+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:57:39.658302+0000","last_undegraded":"2023-10-30T12:57:39.658302+0000","last_fullsized":"2023-10-30T12:57:39.658302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:08.496852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'379","reported_seq":2510,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.669068+0000","last_change":"2023-10-30T12:52:38.960525+0000","last_active":"2023-10-30T12:57:39.669068+0000","last_peered":"2023-10-30T12:57:39.669068+0000","last_clean":"2023-10-30T12:57:39.669068+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:57:39.669068+0000","last_undegraded":"2023-10-30T12:57:39.669068+0000","last_fullsized":"2023-10-30T12:57:39.669068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:09.235440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617272+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:39.617272+0000","last_peered":"2023-10-30T12:57:39.617272+0000","last_clean":"2023-10-30T12:57:39.617272+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:39.617272+0000","last_undegraded":"2023-10-30T12:57:39.617272+0000","last_fullsized":"2023-10-30T12:57:39.617272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:38.878517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'306","reported_seq":2419,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670815+0000","last_change":"2023-10-30T12:52:38.375028+0000","last_active":"2023-10-30T12:57:39.670815+0000","last_peered":"2023-10-30T12:57:39.670815+0000","last_clean":"2023-10-30T12:57:39.670815+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:57:39.670815+0000","last_undegraded":"2023-10-30T12:57:39.670815+0000","last_fullsized":"2023-10-30T12:57:39.670815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:53.865022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'330","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.618194+0000","last_change":"2023-10-30T12:52:38.160968+0000","last_active":"2023-10-30T12:57:39.618194+0000","last_peered":"2023-10-30T12:57:39.618194+0000","last_clean":"2023-10-30T12:57:39.618194+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:39.618194+0000","last_undegraded":"2023-10-30T12:57:39.618194+0000","last_fullsized":"2023-10-30T12:57:39.618194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:14:14.802106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'360","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560965+0000","last_change":"2023-10-30T12:52:38.193081+0000","last_active":"2023-10-30T12:57:21.560965+0000","last_peered":"2023-10-30T12:57:21.560965+0000","last_clean":"2023-10-30T12:57:21.560965+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:57:21.560965+0000","last_undegraded":"2023-10-30T12:57:21.560965+0000","last_fullsized":"2023-10-30T12:57:21.560965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:19.937261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15312207,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":357,"num_write_kb":15140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'376","reported_seq":2497,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642772+0000","last_change":"2023-10-30T12:52:38.708132+0000","last_active":"2023-10-30T12:57:39.642772+0000","last_peered":"2023-10-30T12:57:39.642772+0000","last_clean":"2023-10-30T12:57:39.642772+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:57:39.642772+0000","last_undegraded":"2023-10-30T12:57:39.642772+0000","last_fullsized":"2023-10-30T12:57:39.642772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:23:07.870943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'336","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656156+0000","last_change":"2023-10-30T12:52:38.956531+0000","last_active":"2023-10-30T12:57:39.656156+0000","last_peered":"2023-10-30T12:57:39.656156+0000","last_clean":"2023-10-30T12:57:39.656156+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:57:39.656156+0000","last_undegraded":"2023-10-30T12:57:39.656156+0000","last_fullsized":"2023-10-30T12:57:39.656156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:37.995081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'342","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657965+0000","last_change":"2023-10-30T12:52:38.377927+0000","last_active":"2023-10-30T12:57:39.657965+0000","last_peered":"2023-10-30T12:57:39.657965+0000","last_clean":"2023-10-30T12:57:39.657965+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:39.657965+0000","last_undegraded":"2023-10-30T12:57:39.657965+0000","last_fullsized":"2023-10-30T12:57:39.657965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:29.073770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"36'262","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258065+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:57:12.258065+0000","last_peered":"2023-10-30T12:57:12.258065+0000","last_clean":"2023-10-30T12:57:12.258065+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:12.258065+0000","last_undegraded":"2023-10-30T12:57:12.258065+0000","last_fullsized":"2023-10-30T12:57:12.258065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:08:04.439078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'324","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657884+0000","last_change":"2023-10-30T12:52:38.161345+0000","last_active":"2023-10-30T12:57:39.657884+0000","last_peered":"2023-10-30T12:57:39.657884+0000","last_clean":"2023-10-30T12:57:39.657884+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:39.657884+0000","last_undegraded":"2023-10-30T12:57:39.657884+0000","last_fullsized":"2023-10-30T12:57:39.657884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:18.100051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'281","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.366158+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:57:39.366158+0000","last_peered":"2023-10-30T12:57:39.366158+0000","last_clean":"2023-10-30T12:57:39.366158+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:57:39.366158+0000","last_undegraded":"2023-10-30T12:57:39.366158+0000","last_fullsized":"2023-10-30T12:57:39.366158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403693,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":278,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"52'72","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242321+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:57:12.242321+0000","last_peered":"2023-10-30T12:57:12.242321+0000","last_clean":"2023-10-30T12:57:12.242321+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:12.242321+0000","last_undegraded":"2023-10-30T12:57:12.242321+0000","last_fullsized":"2023-10-30T12:57:12.242321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:47:22.828116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'400","reported_seq":2531,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.613041+0000","last_change":"2023-10-30T12:52:38.375331+0000","last_active":"2023-10-30T12:57:39.613041+0000","last_peered":"2023-10-30T12:57:39.613041+0000","last_clean":"2023-10-30T12:57:39.613041+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:39.613041+0000","last_undegraded":"2023-10-30T12:57:39.613041+0000","last_fullsized":"2023-10-30T12:57:39.613041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:46:29.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"52'341","reported_seq":392,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.262243+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:57:12.262243+0000","last_peered":"2023-10-30T12:57:12.262243+0000","last_clean":"2023-10-30T12:57:12.262243+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:12.262243+0000","last_undegraded":"2023-10-30T12:57:12.262243+0000","last_fullsized":"2023-10-30T12:57:12.262243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'356","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617014+0000","last_change":"2023-10-30T12:52:38.157348+0000","last_active":"2023-10-30T12:57:39.617014+0000","last_peered":"2023-10-30T12:57:39.617014+0000","last_clean":"2023-10-30T12:57:39.617014+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:57:39.617014+0000","last_undegraded":"2023-10-30T12:57:39.617014+0000","last_fullsized":"2023-10-30T12:57:39.617014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:43:06.624515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'31","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:29.786928+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:57:29.786928+0000","last_peered":"2023-10-30T12:57:29.786928+0000","last_clean":"2023-10-30T12:57:29.786928+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:29.786928+0000","last_undegraded":"2023-10-30T12:57:29.786928+0000","last_fullsized":"2023-10-30T12:57:29.786928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":31,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'350","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571444+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:21.571444+0000","last_peered":"2023-10-30T12:57:21.571444+0000","last_clean":"2023-10-30T12:57:21.571444+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:21.571444+0000","last_undegraded":"2023-10-30T12:57:21.571444+0000","last_fullsized":"2023-10-30T12:57:21.571444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:09.875012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:24.827512+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:57:24.827512+0000","last_peered":"2023-10-30T12:57:24.827512+0000","last_clean":"2023-10-30T12:57:24.827512+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:57:24.827512+0000","last_undegraded":"2023-10-30T12:57:24.827512+0000","last_fullsized":"2023-10-30T12:57:24.827512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069734,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":1048,"num_write":70,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'349","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.622444+0000","last_change":"2023-10-30T12:52:38.162408+0000","last_active":"2023-10-30T12:57:39.622444+0000","last_peered":"2023-10-30T12:57:39.622444+0000","last_clean":"2023-10-30T12:57:39.622444+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:57:39.622444+0000","last_undegraded":"2023-10-30T12:57:39.622444+0000","last_fullsized":"2023-10-30T12:57:39.622444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:35:14.339468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"52'43","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.246572+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:57:12.246572+0000","last_peered":"2023-10-30T12:57:12.246572+0000","last_clean":"2023-10-30T12:57:12.246572+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:12.246572+0000","last_undegraded":"2023-10-30T12:57:12.246572+0000","last_fullsized":"2023-10-30T12:57:12.246572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:21:17.143577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'332","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674677+0000","last_change":"2023-10-30T12:52:38.378836+0000","last_active":"2023-10-30T12:57:39.674677+0000","last_peered":"2023-10-30T12:57:39.674677+0000","last_clean":"2023-10-30T12:57:39.674677+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:39.674677+0000","last_undegraded":"2023-10-30T12:57:39.674677+0000","last_fullsized":"2023-10-30T12:57:39.674677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:57:03.390521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'107","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:31.407376+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:57:31.407376+0000","last_peered":"2023-10-30T12:57:31.407376+0000","last_clean":"2023-10-30T12:57:31.407376+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:31.407376+0000","last_undegraded":"2023-10-30T12:57:31.407376+0000","last_fullsized":"2023-10-30T12:57:31.407376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":106,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'306","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573471+0000","last_change":"2023-10-30T12:52:38.163983+0000","last_active":"2023-10-30T12:57:21.573471+0000","last_peered":"2023-10-30T12:57:21.573471+0000","last_clean":"2023-10-30T12:57:21.573471+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:57:21.573471+0000","last_undegraded":"2023-10-30T12:57:21.573471+0000","last_fullsized":"2023-10-30T12:57:21.573471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:26.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"52'354","reported_seq":416,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608375+0000","last_change":"2023-10-30T12:45:18.112894+0000","last_active":"2023-10-30T12:57:07.608375+0000","last_peered":"2023-10-30T12:57:07.608375+0000","last_clean":"2023-10-30T12:57:07.608375+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:07.608375+0000","last_undegraded":"2023-10-30T12:57:07.608375+0000","last_fullsized":"2023-10-30T12:57:07.608375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:29.871295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'359","reported_seq":2485,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.661402+0000","last_change":"2023-10-30T12:52:38.161686+0000","last_active":"2023-10-30T12:57:39.661402+0000","last_peered":"2023-10-30T12:57:39.661402+0000","last_clean":"2023-10-30T12:57:39.661402+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:39.661402+0000","last_undegraded":"2023-10-30T12:57:39.661402+0000","last_fullsized":"2023-10-30T12:57:39.661402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:09:52.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"52'104","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.611180+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:57:07.611180+0000","last_peered":"2023-10-30T12:57:07.611180+0000","last_clean":"2023-10-30T12:57:07.611180+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:07.611180+0000","last_undegraded":"2023-10-30T12:57:07.611180+0000","last_fullsized":"2023-10-30T12:57:07.611180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":7385,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'345","reported_seq":2482,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.571576+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:21.571576+0000","last_peered":"2023-10-30T12:57:21.571576+0000","last_clean":"2023-10-30T12:57:21.571576+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:21.571576+0000","last_undegraded":"2023-10-30T12:57:21.571576+0000","last_fullsized":"2023-10-30T12:57:21.571576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:51.506289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'394","reported_seq":453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.333732+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:57:39.333732+0000","last_peered":"2023-10-30T12:57:39.333732+0000","last_clean":"2023-10-30T12:57:39.333732+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:57:39.333732+0000","last_undegraded":"2023-10-30T12:57:39.333732+0000","last_fullsized":"2023-10-30T12:57:39.333732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16248710,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":387,"num_write_kb":16075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'330","reported_seq":2447,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656223+0000","last_change":"2023-10-30T12:52:38.959038+0000","last_active":"2023-10-30T12:57:39.656223+0000","last_peered":"2023-10-30T12:57:39.656223+0000","last_clean":"2023-10-30T12:57:39.656223+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:57:39.656223+0000","last_undegraded":"2023-10-30T12:57:39.656223+0000","last_fullsized":"2023-10-30T12:57:39.656223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:03:23.132047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"52'112","reported_seq":181,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.608639+0000","last_change":"2023-10-30T12:45:18.075211+0000","last_active":"2023-10-30T12:57:07.608639+0000","last_peered":"2023-10-30T12:57:07.608639+0000","last_clean":"2023-10-30T12:57:07.608639+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:07.608639+0000","last_undegraded":"2023-10-30T12:57:07.608639+0000","last_fullsized":"2023-10-30T12:57:07.608639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:56:42.395200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'357","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.739802+0000","last_change":"2023-10-30T12:52:38.707521+0000","last_active":"2023-10-30T12:57:39.739802+0000","last_peered":"2023-10-30T12:57:39.739802+0000","last_clean":"2023-10-30T12:57:39.739802+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:57:39.739802+0000","last_undegraded":"2023-10-30T12:57:39.739802+0000","last_fullsized":"2023-10-30T12:57:39.739802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:24.903914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'102","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:14.887635+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:57:14.887635+0000","last_peered":"2023-10-30T12:57:14.887635+0000","last_clean":"2023-10-30T12:57:14.887635+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:14.887635+0000","last_undegraded":"2023-10-30T12:57:14.887635+0000","last_fullsized":"2023-10-30T12:57:14.887635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":176,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'304","reported_seq":2424,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.573853+0000","last_change":"2023-10-30T12:52:38.166069+0000","last_active":"2023-10-30T12:57:21.573853+0000","last_peered":"2023-10-30T12:57:21.573853+0000","last_clean":"2023-10-30T12:57:21.573853+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:57:21.573853+0000","last_undegraded":"2023-10-30T12:57:21.573853+0000","last_fullsized":"2023-10-30T12:57:21.573853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:36.932929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12071184,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":306,"num_write_kb":15946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"52'165","reported_seq":217,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745677+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:56:34.745677+0000","last_peered":"2023-10-30T12:56:34.745677+0000","last_clean":"2023-10-30T12:56:34.745677+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:56:34.745677+0000","last_undegraded":"2023-10-30T12:56:34.745677+0000","last_fullsized":"2023-10-30T12:56:34.745677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:19:56.573873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'319","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.574457+0000","last_change":"2023-10-30T12:52:38.167701+0000","last_active":"2023-10-30T12:57:21.574457+0000","last_peered":"2023-10-30T12:57:21.574457+0000","last_clean":"2023-10-30T12:57:21.574457+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:57:21.574457+0000","last_undegraded":"2023-10-30T12:57:21.574457+0000","last_fullsized":"2023-10-30T12:57:21.574457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:54.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"52'123","reported_seq":174,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.787653+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:56:34.787653+0000","last_peered":"2023-10-30T12:56:34.787653+0000","last_clean":"2023-10-30T12:56:34.787653+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:56:34.787653+0000","last_undegraded":"2023-10-30T12:56:34.787653+0000","last_fullsized":"2023-10-30T12:56:34.787653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:03:46.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'357","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558014+0000","last_change":"2023-10-30T12:52:38.194662+0000","last_active":"2023-10-30T12:57:21.558014+0000","last_peered":"2023-10-30T12:57:21.558014+0000","last_clean":"2023-10-30T12:57:21.558014+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:57:21.558014+0000","last_undegraded":"2023-10-30T12:57:21.558014+0000","last_fullsized":"2023-10-30T12:57:21.558014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:33:24.813894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526551,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":357,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"52'12","reported_seq":70,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.745646+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:56:34.745646+0000","last_peered":"2023-10-30T12:56:34.745646+0000","last_clean":"2023-10-30T12:56:34.745646+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:56:34.745646+0000","last_undegraded":"2023-10-30T12:56:34.745646+0000","last_fullsized":"2023-10-30T12:56:34.745646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:32.649920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'323","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.570633+0000","last_change":"2023-10-30T12:52:38.166263+0000","last_active":"2023-10-30T12:57:21.570633+0000","last_peered":"2023-10-30T12:57:21.570633+0000","last_clean":"2023-10-30T12:57:21.570633+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:57:21.570633+0000","last_undegraded":"2023-10-30T12:57:21.570633+0000","last_fullsized":"2023-10-30T12:57:21.570633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:44.813227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468332,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":322,"num_write_kb":15461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.243330+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:57:07.243330+0000","last_peered":"2023-10-30T12:57:07.243330+0000","last_clean":"2023-10-30T12:57:07.243330+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:07.243330+0000","last_undegraded":"2023-10-30T12:57:07.243330+0000","last_fullsized":"2023-10-30T12:57:07.243330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'10","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.242909+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:57:12.242909+0000","last_peered":"2023-10-30T12:57:12.242909+0000","last_clean":"2023-10-30T12:57:12.242909+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:12.242909+0000","last_undegraded":"2023-10-30T12:57:12.242909+0000","last_fullsized":"2023-10-30T12:57:12.242909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'310","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.608876+0000","last_change":"2023-10-30T12:52:38.374620+0000","last_active":"2023-10-30T12:57:39.608876+0000","last_peered":"2023-10-30T12:57:39.608876+0000","last_clean":"2023-10-30T12:57:39.608876+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:57:39.608876+0000","last_undegraded":"2023-10-30T12:57:39.608876+0000","last_fullsized":"2023-10-30T12:57:39.608876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:36:22.529147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"52'497","reported_seq":557,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.260199+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:57:12.260199+0000","last_peered":"2023-10-30T12:57:12.260199+0000","last_clean":"2023-10-30T12:57:12.260199+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:12.260199+0000","last_undegraded":"2023-10-30T12:57:12.260199+0000","last_fullsized":"2023-10-30T12:57:12.260199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'327","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.663068+0000","last_change":"2023-10-30T12:52:38.159894+0000","last_active":"2023-10-30T12:57:39.663068+0000","last_peered":"2023-10-30T12:57:39.663068+0000","last_clean":"2023-10-30T12:57:39.663068+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:57:39.663068+0000","last_undegraded":"2023-10-30T12:57:39.663068+0000","last_fullsized":"2023-10-30T12:57:39.663068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:38:01.190957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'139","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.258850+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:57:12.258850+0000","last_peered":"2023-10-30T12:57:12.258850+0000","last_clean":"2023-10-30T12:57:12.258850+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:57:12.258850+0000","last_undegraded":"2023-10-30T12:57:12.258850+0000","last_fullsized":"2023-10-30T12:57:12.258850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:57:10.795364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4302161,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":135,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'325","reported_seq":2452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617476+0000","last_change":"2023-10-30T12:52:38.157754+0000","last_active":"2023-10-30T12:57:39.617476+0000","last_peered":"2023-10-30T12:57:39.617476+0000","last_clean":"2023-10-30T12:57:39.617476+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:57:39.617476+0000","last_undegraded":"2023-10-30T12:57:39.617476+0000","last_fullsized":"2023-10-30T12:57:39.617476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:15:05.869155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"52'245","reported_seq":297,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.453118+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:56:35.453118+0000","last_peered":"2023-10-30T12:56:35.453118+0000","last_clean":"2023-10-30T12:56:35.453118+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:56:35.453118+0000","last_undegraded":"2023-10-30T12:56:35.453118+0000","last_fullsized":"2023-10-30T12:56:35.453118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'373","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.654586+0000","last_change":"2023-10-30T12:52:38.380527+0000","last_active":"2023-10-30T12:57:39.654586+0000","last_peered":"2023-10-30T12:57:39.654586+0000","last_clean":"2023-10-30T12:57:39.654586+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:39.654586+0000","last_undegraded":"2023-10-30T12:57:39.654586+0000","last_fullsized":"2023-10-30T12:57:39.654586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:28:52.581209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'3","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.239512+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:57:07.239512+0000","last_peered":"2023-10-30T12:57:07.239512+0000","last_clean":"2023-10-30T12:57:07.239512+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:57:07.239512+0000","last_undegraded":"2023-10-30T12:57:07.239512+0000","last_fullsized":"2023-10-30T12:57:07.239512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:40:07.233870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'315","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558514+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:21.558514+0000","last_peered":"2023-10-30T12:57:21.558514+0000","last_clean":"2023-10-30T12:57:21.558514+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:21.558514+0000","last_undegraded":"2023-10-30T12:57:21.558514+0000","last_fullsized":"2023-10-30T12:57:21.558514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:35.262822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9696989,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":311,"num_write_kb":13726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'9","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.243063+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:57:12.243063+0000","last_peered":"2023-10-30T12:57:12.243063+0000","last_clean":"2023-10-30T12:57:12.243063+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:12.243063+0000","last_undegraded":"2023-10-30T12:57:12.243063+0000","last_fullsized":"2023-10-30T12:57:12.243063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'341","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617288+0000","last_change":"2023-10-30T12:52:38.158164+0000","last_active":"2023-10-30T12:57:39.617288+0000","last_peered":"2023-10-30T12:57:39.617288+0000","last_clean":"2023-10-30T12:57:39.617288+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:57:39.617288+0000","last_undegraded":"2023-10-30T12:57:39.617288+0000","last_fullsized":"2023-10-30T12:57:39.617288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:42.652706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'99","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:26.427271+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:57:26.427271+0000","last_peered":"2023-10-30T12:57:26.427271+0000","last_clean":"2023-10-30T12:57:26.427271+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:57:26.427271+0000","last_undegraded":"2023-10-30T12:57:26.427271+0000","last_fullsized":"2023-10-30T12:57:26.427271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:26.913484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":238,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'314","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.655926+0000","last_change":"2023-10-30T12:52:38.374909+0000","last_active":"2023-10-30T12:57:39.655926+0000","last_peered":"2023-10-30T12:57:39.655926+0000","last_clean":"2023-10-30T12:57:39.655926+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:57:39.655926+0000","last_undegraded":"2023-10-30T12:57:39.655926+0000","last_fullsized":"2023-10-30T12:57:39.655926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:27:11.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"52'12","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610156+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:57:07.610156+0000","last_peered":"2023-10-30T12:57:07.610156+0000","last_clean":"2023-10-30T12:57:07.610156+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:07.610156+0000","last_undegraded":"2023-10-30T12:57:07.610156+0000","last_fullsized":"2023-10-30T12:57:07.610156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'331","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617128+0000","last_change":"2023-10-30T12:52:38.161992+0000","last_active":"2023-10-30T12:57:39.617128+0000","last_peered":"2023-10-30T12:57:39.617128+0000","last_clean":"2023-10-30T12:57:39.617128+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:39.617128+0000","last_undegraded":"2023-10-30T12:57:39.617128+0000","last_fullsized":"2023-10-30T12:57:39.617128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:09:36.903572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'163","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.279523+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:57:12.279523+0000","last_peered":"2023-10-30T12:57:12.279523+0000","last_clean":"2023-10-30T12:57:12.279523+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:12.279523+0000","last_undegraded":"2023-10-30T12:57:12.279523+0000","last_fullsized":"2023-10-30T12:57:12.279523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'340","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668698+0000","last_change":"2023-10-30T12:52:38.380106+0000","last_active":"2023-10-30T12:57:39.668698+0000","last_peered":"2023-10-30T12:57:39.668698+0000","last_clean":"2023-10-30T12:57:39.668698+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:39.668698+0000","last_undegraded":"2023-10-30T12:57:39.668698+0000","last_fullsized":"2023-10-30T12:57:39.668698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:12:33.391336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"52'58","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.324062+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:57:35.324062+0000","last_peered":"2023-10-30T12:57:35.324062+0000","last_clean":"2023-10-30T12:57:35.324062+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:57:35.324062+0000","last_undegraded":"2023-10-30T12:57:35.324062+0000","last_fullsized":"2023-10-30T12:57:35.324062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:29:26.355773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'327","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.579159+0000","last_change":"2023-10-30T12:52:38.707846+0000","last_active":"2023-10-30T12:57:39.579159+0000","last_peered":"2023-10-30T12:57:39.579159+0000","last_clean":"2023-10-30T12:57:39.579159+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:57:39.579159+0000","last_undegraded":"2023-10-30T12:57:39.579159+0000","last_fullsized":"2023-10-30T12:57:39.579159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:38:49.638944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'118","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.416434+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:57:19.416434+0000","last_peered":"2023-10-30T12:57:19.416434+0000","last_clean":"2023-10-30T12:57:19.416434+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:57:19.416434+0000","last_undegraded":"2023-10-30T12:57:19.416434+0000","last_fullsized":"2023-10-30T12:57:19.416434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:11:37.720618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":197,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'330","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602487+0000","last_change":"2023-10-30T12:52:38.373796+0000","last_active":"2023-10-30T12:57:39.602487+0000","last_peered":"2023-10-30T12:57:39.602487+0000","last_clean":"2023-10-30T12:57:39.602487+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:57:39.602487+0000","last_undegraded":"2023-10-30T12:57:39.602487+0000","last_fullsized":"2023-10-30T12:57:39.602487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:16.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"52'187","reported_seq":241,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:34.833028+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:56:34.833028+0000","last_peered":"2023-10-30T12:56:34.833028+0000","last_clean":"2023-10-30T12:56:34.833028+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:56:34.833028+0000","last_undegraded":"2023-10-30T12:56:34.833028+0000","last_fullsized":"2023-10-30T12:56:34.833028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":4104,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'332","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659101+0000","last_change":"2023-10-30T12:52:38.955966+0000","last_active":"2023-10-30T12:57:39.659101+0000","last_peered":"2023-10-30T12:57:39.659101+0000","last_clean":"2023-10-30T12:57:39.659101+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:57:39.659101+0000","last_undegraded":"2023-10-30T12:57:39.659101+0000","last_fullsized":"2023-10-30T12:57:39.659101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:55.980109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'314","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.558152+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:21.558152+0000","last_peered":"2023-10-30T12:57:21.558152+0000","last_clean":"2023-10-30T12:57:21.558152+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:21.558152+0000","last_undegraded":"2023-10-30T12:57:21.558152+0000","last_fullsized":"2023-10-30T12:57:21.558152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:52.978943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22515664,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":311,"num_write_kb":26246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"52'151","reported_seq":202,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.983897+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:56:29.983897+0000","last_peered":"2023-10-30T12:56:29.983897+0000","last_clean":"2023-10-30T12:56:29.983897+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:56:29.983897+0000","last_undegraded":"2023-10-30T12:56:29.983897+0000","last_fullsized":"2023-10-30T12:56:29.983897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'315","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:21.560674+0000","last_change":"2023-10-30T12:52:38.194210+0000","last_active":"2023-10-30T12:57:21.560674+0000","last_peered":"2023-10-30T12:57:21.560674+0000","last_clean":"2023-10-30T12:57:21.560674+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:57:21.560674+0000","last_undegraded":"2023-10-30T12:57:21.560674+0000","last_fullsized":"2023-10-30T12:57:21.560674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:56:10.108457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242644,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":313,"num_write_kb":14254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"52'362","reported_seq":420,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.158457+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:56:40.158457+0000","last_peered":"2023-10-30T12:56:40.158457+0000","last_clean":"2023-10-30T12:56:40.158457+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:56:40.158457+0000","last_undegraded":"2023-10-30T12:56:40.158457+0000","last_fullsized":"2023-10-30T12:56:40.158457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'335","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642959+0000","last_change":"2023-10-30T12:52:38.162371+0000","last_active":"2023-10-30T12:57:39.642959+0000","last_peered":"2023-10-30T12:57:39.642959+0000","last_clean":"2023-10-30T12:57:39.642959+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:57:39.642959+0000","last_undegraded":"2023-10-30T12:57:39.642959+0000","last_fullsized":"2023-10-30T12:57:39.642959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:05:58.391344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"52'278","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610628+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:57:07.610628+0000","last_peered":"2023-10-30T12:57:07.610628+0000","last_clean":"2023-10-30T12:57:07.610628+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:07.610628+0000","last_undegraded":"2023-10-30T12:57:07.610628+0000","last_fullsized":"2023-10-30T12:57:07.610628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:49:38.315871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'331","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638149+0000","last_change":"2023-10-30T12:52:38.373954+0000","last_active":"2023-10-30T12:57:39.638149+0000","last_peered":"2023-10-30T12:57:39.638149+0000","last_clean":"2023-10-30T12:57:39.638149+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:57:39.638149+0000","last_undegraded":"2023-10-30T12:57:39.638149+0000","last_fullsized":"2023-10-30T12:57:39.638149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:33:21.106315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"52'193","reported_seq":257,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:14.204972+0000","last_change":"2023-10-30T12:45:18.119848+0000","last_active":"2023-10-30T12:56:14.204972+0000","last_peered":"2023-10-30T12:56:14.204972+0000","last_clean":"2023-10-30T12:56:14.204972+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:56:14.204972+0000","last_undegraded":"2023-10-30T12:56:14.204972+0000","last_fullsized":"2023-10-30T12:56:14.204972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:09:25.732463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925148655,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20870,"num_write_kb":1047645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1900888064,"data_stored":1847301778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21588,"ondisk_log_size":21588,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":174815753,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":580,"num_read_kb":143240,"num_write":5366,"num_write_kb":176360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350330880,"data_stored":350174933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5098,"ondisk_log_size":5098,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405832,"kb_used_data":326080,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93966008,"statfs":{"total":96636764160,"available":96221192192,"internally_reserved":0,"allocated":333905920,"data_stored":326089857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":81656267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.01}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":0.78400000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":1.272},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.073}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.399}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.327},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":1.4870000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281376,"kb_used_data":236312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94090464,"statfs":{"total":96636764160,"available":96348635136,"internally_reserved":0,"allocated":241983488,"data_stored":236763651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.624}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.748}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.1319999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1859999999999999},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359668,"kb_used_data":292396,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94012172,"statfs":{"total":96636764160,"available":96268464128,"internally_reserved":0,"allocated":299413504,"data_stored":291703362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":1.046}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":1.121}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":1.147}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.008}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.75}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.2689999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416500,"kb_used_data":340908,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93955340,"statfs":{"total":96636764160,"available":96210268160,"internally_reserved":0,"allocated":349089792,"data_stored":340928408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":0.84299999999999997}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.753},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89700000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":0.999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.0529999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406132,"kb_used_data":334700,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93965708,"statfs":{"total":96636764160,"available":96220884992,"internally_reserved":0,"allocated":342732800,"data_stored":334429120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73136584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.60599999999999998,"5min":0.54100000000000004,"15min":0.499},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.786,"5min":1.867,"15min":1.867},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":176240,"kb_used_data":149224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94195600,"statfs":{"total":96636764160,"available":96456294400,"internally_reserved":0,"allocated":152805376,"data_stored":147769036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.496},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.68600000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266192,"kb_used_data":212808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94105648,"statfs":{"total":96636764160,"available":96364183552,"internally_reserved":0,"allocated":217915392,"data_stored":212359979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.92900000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":0.85199999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381824,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990016,"statfs":{"total":96636764160,"available":96245776384,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.374},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.45200000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12713984,"data_stored":12691497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42254336,"data_stored":42233706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":60223488,"data_stored":60206961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":34648064,"data_stored":34625544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49438720,"data_stored":49419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264155136,"data_stored":256775105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":261382144,"data_stored":253892880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:41.168 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:57:41.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 Processor -- start 2023-10-30T12:57:41.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 -- start start 2023-10-30T12:57:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4ce00c5c10 0x7f4ce006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce006a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce015cd50 con 0x7f4ce0159760 2023-10-30T12:57:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce015d0a0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.342+0000 7f4ce5e6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00ca000 con 0x7f4ce0069e90 2023-10-30T12:57:41.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4ce00c5c10 0x7f4ce006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f4ce00c5c10 0x7f4ce006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48126/0 (socket says 172.21.15.8:48126) 2023-10-30T12:57:41.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 -- 172.21.15.8:0/3331697684 learned_addr learned my addr 172.21.15.8:0/3331697684 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:41.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdffff640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 -- 172.21.15.8:0/3331697684 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 msgr2=0x7f4ce006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:41.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 -- 172.21.15.8:0/3331697684 >> v1:172.21.15.8:6789/0 conn(0x7f4ce0159760 legacy=0x7f4ce015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 -- 172.21.15.8:0/3331697684 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0159cf0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cdf7fe640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce006eef0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009870 tx=0x7f4cd001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836de117b7f469a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:41.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cde7fc640 1 -- 172.21.15.8:0/3331697684 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00aa670 con 0x7f4ce00c5c10 2023-10-30T12:57:41.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cde7fc640 1 -- 172.21.15.8:0/3331697684 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd00aa7d0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.346+0000 7f4cde7fc640 1 -- 172.21.15.8:0/3331697684 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00aab60 con 0x7f4ce00c5c10 2023-10-30T12:57:41.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/3331697684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 msgr2=0x7f4ce006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce006eef0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009870 tx=0x7f4cd001c510 comp rx=0 tx=0).stop 2023-10-30T12:57:41.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/3331697684 shutdown_connections 2023-10-30T12:57:41.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/3331697684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce006eef0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/3331697684 >> 172.21.15.8:0/3331697684 conn(0x7f4ce006afa0 msgr2=0x7f4ce006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:41.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/3331697684 shutdown_connections 2023-10-30T12:57:41.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/3331697684 wait complete. 2023-10-30T12:57:41.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 Processor -- start 2023-10-30T12:57:41.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- start start 2023-10-30T12:57:41.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce015c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce0159760 0x7f4ce00c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0157410 con 0x7f4ce00c5c10 2023-10-30T12:57:41.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01586c0 con 0x7f4ce0159760 2023-10-30T12:57:41.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4ce5e6b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00ca300 con 0x7f4ce0069e90 2023-10-30T12:57:41.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48138/0 (socket says 172.21.15.8:48138) 2023-10-30T12:57:41.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4cdeffd640 1 -- 172.21.15.8:0/1473235511 learned_addr learned my addr 172.21.15.8:0/1473235511 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:41.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4cdf7fe640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.350+0000 7f4cdffff640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce0159760 0x7f4ce00c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 -- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 msgr2=0x7f4ce015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce015c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 -- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce0159760 msgr2=0x7f4ce00c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce0159760 0x7f4ce00c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 -- 172.21.15.8:0/1473235511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00094d0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cdeffd640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c530 tx=0x7f4cd400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:41.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4005220 con 0x7f4ce00c5c10 2023-10-30T12:57:41.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd4005380 con 0x7f4ce00c5c10 2023-10-30T12:57:41.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4007270 con 0x7f4ce00c5c10 2023-10-30T12:57:41.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce0157620 con 0x7f4ce00c5c10 2023-10-30T12:57:41.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce0157be0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce004e170 con 0x7f4ce00c5c10 2023-10-30T12:57:41.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4cd40073d0 con 0x7f4ce00c5c10 2023-10-30T12:57:41.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.354+0000 7f4cc7fff640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 0x7f4ccc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:41.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.358+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f4cd4083a30 con 0x7f4ce00c5c10 2023-10-30T12:57:41.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.358+0000 7f4cdf7fe640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 0x7f4ccc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:41.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.358+0000 7f4cdf7fe640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 0x7f4ccc068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00ae000 tx=0x7f4cd00a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:41.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.358+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4cd4050c00 con 0x7f4ce00c5c10 2023-10-30T12:57:41.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.522+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce006b700 con 0x7f4ce00c5c10 2023-10-30T12:57:41.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.522+0000 7f4cc7fff640 1 -- 172.21.15.8:0/1473235511 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17471 (secure 0 0 0) 0x7f4cd4054220 con 0x7f4ce00c5c10 2023-10-30T12:57:41.530 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:41.530 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":53,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:57:07.233484+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6838/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.8:6839/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.8:6837/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.46:6833/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.46:6832/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6838/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6839/710307090":"2023-10-31T12:47:22.043669+0000","172.21.15.8:6834/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/710307090":"2023-10-31T12:47:22.043669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:57:41.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 msgr2=0x7f4ccc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 0x7f4ccc068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00ae000 tx=0x7f4cd00a7000 comp rx=0 tx=0).stop 2023-10-30T12:57:41.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 msgr2=0x7f4ce00c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:41.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c530 tx=0x7f4cd400c9f0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 shutdown_connections 2023-10-30T12:57:41.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f4ccc0664a0 0x7f4ccc068960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce0159760 0x7f4ce00c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ce00c5c10 0x7f4ce00c37c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 --2- 172.21.15.8:0/1473235511 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f4ce0069e90 0x7f4ce015c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:41.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 >> 172.21.15.8:0/1473235511 conn(0x7f4ce006afa0 msgr2=0x7f4ce0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:41.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 shutdown_connections 2023-10-30T12:57:41.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:41.526+0000 7f4ce5e6b640 1 -- 172.21.15.8:0/1473235511 wait complete. 2023-10-30T12:57:42.554 INFO:tasks.ceph:Scrubbing osd.0 2023-10-30T12:57:42.555 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:42.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 Processor -- start 2023-10-30T12:57:42.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 -- start start 2023-10-30T12:57:42.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1bd80c4340 0x7f1bd81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd806af00 con 0x7f1bd80c4720 2023-10-30T12:57:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd806a660 con 0x7f1bd80ca470 2023-10-30T12:57:42.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdfe54640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd80c4cd0 con 0x7f1bd80c4340 2023-10-30T12:57:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1bd80c4340 0x7f1bd81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1bd80c4340 0x7f1bd81599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48148/0 (socket says 172.21.15.8:48148) 2023-10-30T12:57:42.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 -- 172.21.15.8:0/2586215445 learned_addr learned my addr 172.21.15.8:0/2586215445 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdd3ef640 1 --1- 172.21.15.8:0/2586215445 >> v1:172.21.15.8:6789/0 conn(0x7f1bd80c4720 0x7f1bd8069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37984/0 (socket says 172.21.15.8:37984) 2023-10-30T12:57:42.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 -- 172.21.15.8:0/2586215445 >> v1:172.21.15.46:6789/0 conn(0x7f1bd80ca470 legacy=0x7f1bd8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:42.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 -- 172.21.15.8:0/2586215445 >> v1:172.21.15.8:6789/0 conn(0x7f1bd80c4720 legacy=0x7f1bd8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bddbf0640 1 -- 172.21.15.8:0/2586215445 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd806a940 con 0x7f1bd80c4340 2023-10-30T12:57:42.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.710+0000 7f1bdcbee640 1 -- 172.21.15.8:0/2586215445 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994027577 0 0) 0x7f1bd806af00 con 0x7f1bd80c4720 2023-10-30T12:57:42.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bddbf0640 1 --2- 172.21.15.8:0/2586215445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4340 0x7f1bd81599e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009870 tx=0x7f1bd401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c973f9ac96da7b server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdcbee640 1 -- 172.21.15.8:0/2586215445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4004230 con 0x7f1bd80c4340 2023-10-30T12:57:42.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdcbee640 1 -- 172.21.15.8:0/2586215445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd40a7050 con 0x7f1bd80c4340 2023-10-30T12:57:42.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdcbee640 1 -- 172.21.15.8:0/2586215445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4003620 con 0x7f1bd80c4340 2023-10-30T12:57:42.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 -- 172.21.15.8:0/2586215445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4340 msgr2=0x7f1bd81599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 --2- 172.21.15.8:0/2586215445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4340 0x7f1bd81599e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009870 tx=0x7f1bd401c440 comp rx=0 tx=0).stop 2023-10-30T12:57:42.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 -- 172.21.15.8:0/2586215445 shutdown_connections 2023-10-30T12:57:42.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 --2- 172.21.15.8:0/2586215445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4340 0x7f1bd81599e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 -- 172.21.15.8:0/2586215445 >> 172.21.15.8:0/2586215445 conn(0x7f1bd80cf1c0 msgr2=0x7f1bd80cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:42.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 -- 172.21.15.8:0/2586215445 shutdown_connections 2023-10-30T12:57:42.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.714+0000 7f1bdfe54640 1 -- 172.21.15.8:0/2586215445 wait complete. 2023-10-30T12:57:42.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 Processor -- start 2023-10-30T12:57:42.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- start start 2023-10-30T12:57:42.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd80c4340 0x7f1bd814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 0x7f1bd8147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1bd80ca470 0x7f1bd8147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81592c0 con 0x7f1bd80c4720 2023-10-30T12:57:42.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8142e10 con 0x7f1bd80c4340 2023-10-30T12:57:42.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8142520 con 0x7f1bd80ca470 2023-10-30T12:57:42.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 0x7f1bd8147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 0x7f1bd8147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48158/0 (socket says 172.21.15.8:48158) 2023-10-30T12:57:42.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 -- 172.21.15.8:0/3559800964 learned_addr learned my addr 172.21.15.8:0/3559800964 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:42.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bddbf0640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd80c4340 0x7f1bd814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bde3f1640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1bd80ca470 0x7f1bd8147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 -- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1bd80ca470 msgr2=0x7f1bd8147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1bd80ca470 0x7f1bd8147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 -- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd80c4340 msgr2=0x7f1bd814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd80c4340 0x7f1bd814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd40094d0 con 0x7f1bd80c4720 2023-10-30T12:57:42.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdd3ef640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 0x7f1bd8147240 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c3e0 tx=0x7f1bc800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80074e0 con 0x7f1bd80c4720 2023-10-30T12:57:42.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc8007640 con 0x7f1bd80c4720 2023-10-30T12:57:42.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd8148d30 con 0x7f1bd80c4720 2023-10-30T12:57:42.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8004a30 con 0x7f1bd80c4720 2023-10-30T12:57:42.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.718+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd81427c0 con 0x7f1bd80c4720 2023-10-30T12:57:42.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bd80c9f50 con 0x7f1bd80c4720 2023-10-30T12:57:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1bc8004b90 con 0x7f1bd80c4720 2023-10-30T12:57:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bbc0664a0 0x7f1bbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f1bc8083cb0 con 0x7f1bd80c4720 2023-10-30T12:57:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bddbf0640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bbc0664a0 0x7f1bbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7f1bbc06ec50 0x7f1bbc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bbc0717c0 con 0x7f1bbc06ec50 2023-10-30T12:57:42.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc8086290 con 0x7f1bd80c4720 2023-10-30T12:57:42.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bde3f1640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7f1bbc06ec50 0x7f1bbc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bddbf0640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bbc0664a0 0x7f1bbc068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40ad000 tx=0x7f1bd4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bde3f1640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7f1bbc06ec50 0x7f1bbc0710f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.722+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== osd.0 v2:172.21.15.8:6801/2064691129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1bbc0717c0 con 0x7f1bbc06ec50 2023-10-30T12:57:42.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 --> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1bd80725b0 con 0x7f1bbc06ec50 2023-10-30T12:57:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bce7fc640 1 -- 172.21.15.8:0/3559800964 <== osd.0 v2:172.21.15.8:6801/2064691129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1bd80725b0 con 0x7f1bbc06ec50 2023-10-30T12:57:42.743 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:42.744 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:42.744 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7f1bbc06ec50 msgr2=0x7f1bbc0710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6801/2064691129,v1:172.21.15.8:6805/2064691129] conn(0x7f1bbc06ec50 0x7f1bbc0710f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bbc0664a0 msgr2=0x7f1bbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1bbc0664a0 0x7f1bbc068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40ad000 tx=0x7f1bd4008040 comp rx=0 tx=0).stop 2023-10-30T12:57:42.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 msgr2=0x7f1bd8147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 --2- 172.21.15.8:0/3559800964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bd80c4720 0x7f1bd8147240 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c3e0 tx=0x7f1bc800c8a0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bde3f1640 1 -- 172.21.15.8:0/3559800964 reap_dead start 2023-10-30T12:57:42.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 shutdown_connections 2023-10-30T12:57:42.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 >> 172.21.15.8:0/3559800964 conn(0x7f1bd80cf1c0 msgr2=0x7f1bd8157710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:42.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 shutdown_connections 2023-10-30T12:57:42.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.738+0000 7f1bdfe54640 1 -- 172.21.15.8:0/3559800964 wait complete. 2023-10-30T12:57:42.762 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-30T12:57:42.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 Processor -- start 2023-10-30T12:57:42.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 -- start start 2023-10-30T12:57:42.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 0x7fdae806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 0x7fdae80c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae80c2310 con 0x7fdae80c3e40 2023-10-30T12:57:42.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae8156710 con 0x7fdae8150830 2023-10-30T12:57:42.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdaedfc5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80c4360 con 0x7fdae80cb7d0 2023-10-30T12:57:42.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 0x7fdae806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae6ffd640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 0x7fdae80c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 0x7fdae806b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:50926/0 (socket says 172.21.15.8:50926) 2023-10-30T12:57:42.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae77fe640 1 -- 172.21.15.8:0/2158407553 learned_addr learned my addr 172.21.15.8:0/2158407553 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:42.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae6ffd640 1 -- 172.21.15.8:0/2158407553 >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 msgr2=0x7fdae806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:42.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae6ffd640 1 --2- 172.21.15.8:0/2158407553 >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 0x7fdae806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae6ffd640 1 -- 172.21.15.8:0/2158407553 >> v1:172.21.15.46:6789/0 conn(0x7fdae80c3e40 legacy=0x7fdae8070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.902+0000 7fdae6ffd640 1 -- 172.21.15.8:0/2158407553 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae81569b0 con 0x7fdae80cb7d0 2023-10-30T12:57:42.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdae6ffd640 1 --2- 172.21.15.8:0/2158407553 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 0x7fdae80c3900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097c0 tx=0x7fdadc01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fd4b22e6adb5eb server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdae67fc640 1 -- 172.21.15.8:0/2158407553 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0a8970 con 0x7fdae80cb7d0 2023-10-30T12:57:42.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdae67fc640 1 -- 172.21.15.8:0/2158407553 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0a8ad0 con 0x7fdae80cb7d0 2023-10-30T12:57:42.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdae67fc640 1 -- 172.21.15.8:0/2158407553 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0a8e60 con 0x7fdae80cb7d0 2023-10-30T12:57:42.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 -- 172.21.15.8:0/2158407553 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 msgr2=0x7fdae80c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/2158407553 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 0x7fdae80c3900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0097c0 tx=0x7fdadc01ec70 comp rx=0 tx=0).stop 2023-10-30T12:57:42.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 -- 172.21.15.8:0/2158407553 shutdown_connections 2023-10-30T12:57:42.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/2158407553 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae80cb7d0 0x7fdae80c3900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/2158407553 >> v2:172.21.15.46:3300/0 conn(0x7fdae8150830 0x7fdae806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 -- 172.21.15.8:0/2158407553 >> 172.21.15.8:0/2158407553 conn(0x7fdae80cac70 msgr2=0x7fdae80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:42.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 -- 172.21.15.8:0/2158407553 shutdown_connections 2023-10-30T12:57:42.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.906+0000 7fdaedfc5640 1 -- 172.21.15.8:0/2158407553 wait complete. 2023-10-30T12:57:42.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 Processor -- start 2023-10-30T12:57:42.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- start start 2023-10-30T12:57:42.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae80c3e40 0x7fdae8147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 0x7fdae8141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81efb00 con 0x7fdae80cb7d0 2023-10-30T12:57:42.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81eeca0 con 0x7fdae80c3e40 2023-10-30T12:57:42.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81eef80 con 0x7fdae8150830 2023-10-30T12:57:42.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48172/0 (socket says 172.21.15.8:48172) 2023-10-30T12:57:42.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 -- 172.21.15.8:0/4201992067 learned_addr learned my addr 172.21.15.8:0/4201992067 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:42.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae7fff640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 0x7fdae8141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae77fe640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae80c3e40 0x7fdae8147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 -- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 msgr2=0x7fdae8141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 0x7fdae8141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 -- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae80c3e40 msgr2=0x7fdae8147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:42.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae80c3e40 0x7fdae8147b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:42.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 -- 172.21.15.8:0/4201992067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc009420 con 0x7fdae80cb7d0 2023-10-30T12:57:42.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae7fff640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 0x7fdae8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:42.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdae6ffd640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0b3040 tx=0x7fdadc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b9630 con 0x7fdae80cb7d0 2023-10-30T12:57:42.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0b97b0 con 0x7fdae80cb7d0 2023-10-30T12:57:42.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b9b60 con 0x7fdae80cb7d0 2023-10-30T12:57:42.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae81ef260 con 0x7fdae80cb7d0 2023-10-30T12:57:42.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.910+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae81ef820 con 0x7fdae80cb7d0 2023-10-30T12:57:42.927 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae8066510 con 0x7fdae80cb7d0 2023-10-30T12:57:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdadc0b7040 con 0x7fdae80cb7d0 2023-10-30T12:57:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdacffff640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 0x7fdad4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fdadc1220c0 con 0x7fdae80cb7d0 2023-10-30T12:57:42.928 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdae77fe640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 0x7fdad4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.914+0000 7fdae77fe640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 0x7fdad4068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ab0 tx=0x7fdad8009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:42.929 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:42.918+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdadc0ef200 con 0x7fdae80cb7d0 2023-10-30T12:57:43.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.074+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdae8156f70 con 0x7fdad40664a0 2023-10-30T12:57:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.074+0000 7fdacffff640 1 -- 172.21.15.8:0/4201992067 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdad8011a00 con 0x7fdad40664a0 2023-10-30T12:57:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 0 to deep-scrub 2023-10-30T12:57:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 msgr2=0x7fdad4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 0x7fdad4068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ab0 tx=0x7fdad8009240 comp rx=0 tx=0).stop 2023-10-30T12:57:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 msgr2=0x7fdae8141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0b3040 tx=0x7fdadc01ba10 comp rx=0 tx=0).stop 2023-10-30T12:57:43.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 shutdown_connections 2023-10-30T12:57:43.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdad40664a0 0x7fdad4068960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdae8150830 0x7fdae8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae80cb7d0 0x7fdae8141570 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 --2- 172.21.15.8:0/4201992067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae80c3e40 0x7fdae8147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 >> 172.21.15.8:0/4201992067 conn(0x7fdae80cac70 msgr2=0x7fdae80733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 shutdown_connections 2023-10-30T12:57:43.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.078+0000 7fdaedfc5640 1 -- 172.21.15.8:0/4201992067 wait complete. 2023-10-30T12:57:43.103 INFO:tasks.ceph:Scrubbing osd.1 2023-10-30T12:57:43.104 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 Processor -- start 2023-10-30T12:57:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 -- start start 2023-10-30T12:57:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1330151b10 0x7f13300c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1330157620 con 0x7f1330151730 2023-10-30T12:57:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1330072bd0 con 0x7f1330072740 2023-10-30T12:57:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.246+0000 7f1334ac9640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1330158560 con 0x7f1330151b10 2023-10-30T12:57:43.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132e59c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f1330151730 0x7f13300c5fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37992/0 (socket says 172.21.15.8:37992) 2023-10-30T12:57:43.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132e59c640 1 -- 172.21.15.8:0/2954063799 learned_addr learned my addr 172.21.15.8:0/2954063799 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:43.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f1327fff640 1 --2- 172.21.15.8:0/2954063799 >> v2:172.21.15.8:3300/0 conn(0x7f1330151b10 0x7f13300c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132dd9b640 1 -- 172.21.15.8:0/2954063799 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402705100 0 0) 0x7f1330157620 con 0x7f1330151730 2023-10-30T12:57:43.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132dd9b640 1 -- 172.21.15.8:0/2954063799 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003530 con 0x7f1330151730 2023-10-30T12:57:43.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f1327fff640 1 -- 172.21.15.8:0/2954063799 >> v1:172.21.15.46:6789/0 conn(0x7f1330072740 legacy=0x7f133006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f1327fff640 1 -- 172.21.15.8:0/2954063799 >> v1:172.21.15.8:6789/0 conn(0x7f1330151730 legacy=0x7f13300c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f1327fff640 1 -- 172.21.15.8:0/2954063799 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13301578e0 con 0x7f1330151b10 2023-10-30T12:57:43.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f1327fff640 1 --2- 172.21.15.8:0/2954063799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151b10 0x7f13300c57e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13180097c0 tx=0x7f131801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdb3ae2fc82f5e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132dd9b640 1 -- 172.21.15.8:0/2954063799 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180a8970 con 0x7f1330151b10 2023-10-30T12:57:43.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132dd9b640 1 -- 172.21.15.8:0/2954063799 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13180a8ad0 con 0x7f1330151b10 2023-10-30T12:57:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.250+0000 7f132dd9b640 1 -- 172.21.15.8:0/2954063799 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180a8e60 con 0x7f1330151b10 2023-10-30T12:57:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- 172.21.15.8:0/2954063799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151b10 msgr2=0x7f13300c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 --2- 172.21.15.8:0/2954063799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151b10 0x7f13300c57e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13180097c0 tx=0x7f131801ec70 comp rx=0 tx=0).stop 2023-10-30T12:57:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- 172.21.15.8:0/2954063799 shutdown_connections 2023-10-30T12:57:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 --2- 172.21.15.8:0/2954063799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151b10 0x7f13300c57e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- 172.21.15.8:0/2954063799 >> 172.21.15.8:0/2954063799 conn(0x7f133006e150 msgr2=0x7f1330069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- 172.21.15.8:0/2954063799 shutdown_connections 2023-10-30T12:57:43.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- 172.21.15.8:0/2954063799 wait complete. 2023-10-30T12:57:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 Processor -- start 2023-10-30T12:57:43.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- start start 2023-10-30T12:57:43.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1330072740 0x7f133013f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 0x7f13301e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1330151b10 0x7f13301eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300c0230 con 0x7f1330151730 2023-10-30T12:57:43.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300bff30 con 0x7f1330072740 2023-10-30T12:57:43.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1334ac9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13301f3060 con 0x7f1330151b10 2023-10-30T12:57:43.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 0x7f13301e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 0x7f13301e40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48182/0 (socket says 172.21.15.8:48182) 2023-10-30T12:57:43.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 -- 172.21.15.8:0/2350306919 learned_addr learned my addr 172.21.15.8:0/2350306919 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f132e59c640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1330072740 0x7f133013f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f132ed9d640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1330151b10 0x7f13301eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 -- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1330151b10 msgr2=0x7f13301eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1330151b10 0x7f13301eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 -- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1330072740 msgr2=0x7f133013f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1330072740 0x7f133013f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.254+0000 7f1327fff640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009420 con 0x7f1330151730 2023-10-30T12:57:43.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1327fff640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 0x7f13301e40e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f131801c8a0 tx=0x7f131801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180bb640 con 0x7f1330151730 2023-10-30T12:57:43.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13180bb7c0 con 0x7f1330151730 2023-10-30T12:57:43.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180bbb70 con 0x7f1330151730 2023-10-30T12:57:43.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1330140450 con 0x7f1330151730 2023-10-30T12:57:43.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1330144f40 con 0x7f1330151730 2023-10-30T12:57:43.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1330157ea0 con 0x7f1330151730 2023-10-30T12:57:43.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13180b7040 con 0x7f1330151730 2023-10-30T12:57:43.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13100664a0 0x7f1310068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f1318122f70 con 0x7f1330151730 2023-10-30T12:57:43.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f132e59c640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13100664a0 0x7f1310068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f131006ec50 0x7f13100710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13100717c0 con 0x7f131006ec50 2023-10-30T12:57:43.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13181233c0 con 0x7f1330151730 2023-10-30T12:57:43.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.258+0000 7f132ed9d640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f131006ec50 0x7f13100710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.262+0000 7f132e59c640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13100664a0 0x7f1310068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13200011f0 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.262+0000 7f132ed9d640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f131006ec50 0x7f13100710f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.262+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== osd.1 v2:172.21.15.8:6804/2525970788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f13100717c0 con 0x7f131006ec50 2023-10-30T12:57:43.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 --> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1330157bc0 con 0x7f131006ec50 2023-10-30T12:57:43.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1326ffd640 1 -- 172.21.15.8:0/2350306919 <== osd.1 v2:172.21.15.8:6804/2525970788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1328001de0 con 0x7f131006ec50 2023-10-30T12:57:43.280 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:43.281 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:43.281 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:43.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f131006ec50 msgr2=0x7f13100710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6804/2525970788,v1:172.21.15.8:6807/2525970788] conn(0x7f131006ec50 0x7f13100710f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13100664a0 msgr2=0x7f1310068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f13100664a0 0x7f1310068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13200011f0 tx=0x7f1320009040 comp rx=0 tx=0).stop 2023-10-30T12:57:43.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 msgr2=0x7f13301e40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 --2- 172.21.15.8:0/2350306919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1330151730 0x7f13301e40e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f131801c8a0 tx=0x7f131801ba10 comp rx=0 tx=0).stop 2023-10-30T12:57:43.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f132ed9d640 1 -- 172.21.15.8:0/2350306919 reap_dead start 2023-10-30T12:57:43.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 shutdown_connections 2023-10-30T12:57:43.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 >> 172.21.15.8:0/2350306919 conn(0x7f133006e150 msgr2=0x7f13300c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 shutdown_connections 2023-10-30T12:57:43.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.274+0000 7f1334ac9640 1 -- 172.21.15.8:0/2350306919 wait complete. 2023-10-30T12:57:43.299 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-30T12:57:43.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 Processor -- start 2023-10-30T12:57:43.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 -- start start 2023-10-30T12:57:43.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 0x7f550815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f55080c4a10 0x7f55080c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f550815a3b0 0x7f55080ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550815dec0 con 0x7f55080c4a10 2023-10-30T12:57:43.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55081599a0 con 0x7f5508153c80 2023-10-30T12:57:43.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550f11d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5508159f20 con 0x7f550815a3b0 2023-10-30T12:57:43.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f55080c4a10 0x7f55080c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f55080c4a10 0x7f55080c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48196/0 (socket says 172.21.15.8:48196) 2023-10-30T12:57:43.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 -- 172.21.15.8:0/3894921602 learned_addr learned my addr 172.21.15.8:0/3894921602 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550ceb9640 1 --2- 172.21.15.8:0/3894921602 >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 0x7f550815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550d6ba640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f550815a3b0 0x7f55080ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 -- 172.21.15.8:0/3894921602 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f550815a3b0 msgr2=0x7f55080ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f550815a3b0 0x7f55080ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 -- 172.21.15.8:0/3894921602 >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 msgr2=0x7f550815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 --2- 172.21.15.8:0/3894921602 >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 0x7f550815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f5507fff640 1 -- 172.21.15.8:0/3894921602 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080c47e0 con 0x7f55080c4a10 2023-10-30T12:57:43.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.438+0000 7f550ceb9640 1 --2- 172.21.15.8:0/3894921602 >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 0x7f550815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:43.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f5507fff640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55080c4a10 0x7f55080c42a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55080d00f0 tx=0x7f54f401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51709afc29de21a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f55077fe640 1 -- 172.21.15.8:0/3894921602 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a89b0 con 0x7f55080c4a10 2023-10-30T12:57:43.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f55077fe640 1 -- 172.21.15.8:0/3894921602 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f40a8b10 con 0x7f55080c4a10 2023-10-30T12:57:43.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f55077fe640 1 -- 172.21.15.8:0/3894921602 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a8ea0 con 0x7f55080c4a10 2023-10-30T12:57:43.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 -- 172.21.15.8:0/3894921602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55080c4a10 msgr2=0x7f55080c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55080c4a10 0x7f55080c42a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55080d00f0 tx=0x7f54f401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 -- 172.21.15.8:0/3894921602 shutdown_connections 2023-10-30T12:57:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f550815a3b0 0x7f55080ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 --2- 172.21.15.8:0/3894921602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55080c4a10 0x7f55080c42a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 --2- 172.21.15.8:0/3894921602 >> v2:172.21.15.46:3300/0 conn(0x7f5508153c80 0x7f550815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 -- 172.21.15.8:0/3894921602 >> 172.21.15.8:0/3894921602 conn(0x7f55080715c0 msgr2=0x7f55080719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 -- 172.21.15.8:0/3894921602 shutdown_connections 2023-10-30T12:57:43.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.442+0000 7f550f11d640 1 -- 172.21.15.8:0/3894921602 wait complete. 2023-10-30T12:57:43.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 Processor -- start 2023-10-30T12:57:43.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- start start 2023-10-30T12:57:43.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55080c4a10 0x7f55081f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550815a3b0 0x7f5508147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55081f2c20 con 0x7f5508153c80 2023-10-30T12:57:43.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5508148620 con 0x7f550815a3b0 2023-10-30T12:57:43.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55081ee870 con 0x7f55080c4a10 2023-10-30T12:57:43.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48208/0 (socket says 172.21.15.8:48208) 2023-10-30T12:57:43.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 -- 172.21.15.8:0/4254559573 learned_addr learned my addr 172.21.15.8:0/4254559573 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550ceb9640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55080c4a10 0x7f55081f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550d6ba640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550815a3b0 0x7f5508147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 -- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55080c4a10 msgr2=0x7f55081f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55080c4a10 0x7f55081f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 -- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550815a3b0 msgr2=0x7f5508147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550815a3b0 0x7f5508147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 -- 172.21.15.8:0/4254559573 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40094d0 con 0x7f5508153c80 2023-10-30T12:57:43.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f5507fff640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f54f401d3a0 tx=0x7f54f401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40ba600 con 0x7f5508153c80 2023-10-30T12:57:43.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f40ba760 con 0x7f5508153c80 2023-10-30T12:57:43.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55081eeb50 con 0x7f5508153c80 2023-10-30T12:57:43.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40bab10 con 0x7f5508153c80 2023-10-30T12:57:43.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.446+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081ef110 con 0x7f5508153c80 2023-10-30T12:57:43.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550804e170 con 0x7f5508153c80 2023-10-30T12:57:43.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f54f40bac70 con 0x7f5508153c80 2023-10-30T12:57:43.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f55057fa640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 0x7f54ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f54f41227c0 con 0x7f5508153c80 2023-10-30T12:57:43.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f550ceb9640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 0x7f54ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.450+0000 7f550ceb9640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 0x7f54ec068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00c880 tx=0x7f54fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.454+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f54f40ef900 con 0x7f5508153c80 2023-10-30T12:57:43.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f550815d900 con 0x7f54ec0664a0 2023-10-30T12:57:43.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f55057fa640 1 -- 172.21.15.8:0/4254559573 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f54fc00ce60 con 0x7f54ec0664a0 2023-10-30T12:57:43.619 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 1 to deep-scrub 2023-10-30T12:57:43.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 msgr2=0x7f54ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 0x7f54ec068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00c880 tx=0x7f54fc008040 comp rx=0 tx=0).stop 2023-10-30T12:57:43.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 msgr2=0x7f55080b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f54f401d3a0 tx=0x7f54f401ba90 comp rx=0 tx=0).stop 2023-10-30T12:57:43.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 shutdown_connections 2023-10-30T12:57:43.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f54ec0664a0 0x7f54ec068960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550815a3b0 0x7f5508147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5508153c80 0x7f55080b4d30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 --2- 172.21.15.8:0/4254559573 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f55080c4a10 0x7f55081f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 >> 172.21.15.8:0/4254559573 conn(0x7f55080715c0 msgr2=0x7f55080c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 shutdown_connections 2023-10-30T12:57:43.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.614+0000 7f550f11d640 1 -- 172.21.15.8:0/4254559573 wait complete. 2023-10-30T12:57:43.641 INFO:tasks.ceph:Scrubbing osd.2 2023-10-30T12:57:43.642 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:43.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.782+0000 7fe6d190f640 1 Processor -- start 2023-10-30T12:57:43.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.782+0000 7fe6d190f640 1 -- start start 2023-10-30T12:57:43.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6d190f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe6cc0c4720 0x7fe6cc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6d190f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6cc0ca470 0x7fe6cc0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6d190f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc06af00 con 0x7fe6cc0c4340 2023-10-30T12:57:43.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6d190f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0c4cd0 con 0x7fe6cc0c4720 2023-10-30T12:57:43.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6d190f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc06a660 con 0x7fe6cc0ca470 2023-10-30T12:57:43.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe6cc0c4720 0x7fe6cc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe6cc0c4720 0x7fe6cc1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48214/0 (socket says 172.21.15.8:48214) 2023-10-30T12:57:43.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 -- 172.21.15.8:0/724601645 learned_addr learned my addr 172.21.15.8:0/724601645 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6cb7fe640 1 --2- 172.21.15.8:0/724601645 >> v2:172.21.15.46:3300/0 conn(0x7fe6cc0ca470 0x7fe6cc0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 -- 172.21.15.8:0/724601645 >> v2:172.21.15.46:3300/0 conn(0x7fe6cc0ca470 msgr2=0x7fe6cc0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 --2- 172.21.15.8:0/724601645 >> v2:172.21.15.46:3300/0 conn(0x7fe6cc0ca470 0x7fe6cc0ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 -- 172.21.15.8:0/724601645 >> v1:172.21.15.46:6789/0 conn(0x7fe6cc0c4340 legacy=0x7fe6cc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.786+0000 7fe6c3fff640 1 -- 172.21.15.8:0/724601645 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc06a940 con 0x7fe6cc0c4720 2023-10-30T12:57:43.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6c3fff640 1 --2- 172.21.15.8:0/724601645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1599e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc06bf30 tx=0x7fe6b401c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5cefee75a82d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6ca7fc640 1 -- 172.21.15.8:0/724601645 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40ab600 con 0x7fe6cc0c4720 2023-10-30T12:57:43.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6ca7fc640 1 -- 172.21.15.8:0/724601645 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b40a7050 con 0x7fe6cc0c4720 2023-10-30T12:57:43.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6ca7fc640 1 -- 172.21.15.8:0/724601645 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40ab980 con 0x7fe6cc0c4720 2023-10-30T12:57:43.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- 172.21.15.8:0/724601645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4720 msgr2=0x7fe6cc1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 --2- 172.21.15.8:0/724601645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1599e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc06bf30 tx=0x7fe6b401c180 comp rx=0 tx=0).stop 2023-10-30T12:57:43.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- 172.21.15.8:0/724601645 shutdown_connections 2023-10-30T12:57:43.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 --2- 172.21.15.8:0/724601645 >> v2:172.21.15.46:3300/0 conn(0x7fe6cc0ca470 0x7fe6cc0ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 --2- 172.21.15.8:0/724601645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1599e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- 172.21.15.8:0/724601645 >> 172.21.15.8:0/724601645 conn(0x7fe6cc0cf1c0 msgr2=0x7fe6cc0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- 172.21.15.8:0/724601645 shutdown_connections 2023-10-30T12:57:43.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- 172.21.15.8:0/724601645 wait complete. 2023-10-30T12:57:43.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 Processor -- start 2023-10-30T12:57:43.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.790+0000 7fe6d190f640 1 -- start start 2023-10-30T12:57:43.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 0x7fe6cc156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe6cc0ca470 0x7fe6cc154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc153120 con 0x7fe6cc0c4340 2023-10-30T12:57:43.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc14c7a0 con 0x7fe6cc0c4720 2023-10-30T12:57:43.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc14d120 con 0x7fe6cc0ca470 2023-10-30T12:57:43.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 0x7fe6cc156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 0x7fe6cc156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48224/0 (socket says 172.21.15.8:48224) 2023-10-30T12:57:43.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 -- 172.21.15.8:0/1883821768 learned_addr learned my addr 172.21.15.8:0/1883821768 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6c3fff640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6cb7fe640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe6cc0ca470 0x7fe6cc154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 -- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe6cc0ca470 msgr2=0x7fe6cc154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe6cc0ca470 0x7fe6cc154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 -- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6cc0c4720 msgr2=0x7fe6cc1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6cc0c4720 0x7fe6cc1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40094d0 con 0x7fe6cc0c4340 2023-10-30T12:57:43.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6cb7fe640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe6cc0ca470 0x7fe6cc154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:57:43.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6caffd640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 0x7fe6cc156b10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c3d0 tx=0x7fe6bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0052d0 con 0x7fe6cc0c4340 2023-10-30T12:57:43.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc005450 con 0x7fe6cc0c4340 2023-10-30T12:57:43.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003860 con 0x7fe6cc0c4340 2023-10-30T12:57:43.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc153300 con 0x7fe6cc0c4340 2023-10-30T12:57:43.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc14cdb0 con 0x7fe6cc0c4340 2023-10-30T12:57:43.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6cc0c9f50 con 0x7fe6cc0c4340 2023-10-30T12:57:43.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.794+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe6bc0039c0 con 0x7fe6cc0c4340 2023-10-30T12:57:43.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6ac0664d0 0x7fe6ac068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fe6bc0837a0 con 0x7fe6cc0c4340 2023-10-30T12:57:43.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c3fff640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6ac0664d0 0x7fe6ac068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7fe6ac06ec80 0x7fe6ac071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6ac0717f0 con 0x7fe6ac06ec80 2023-10-30T12:57:43.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6bc085d20 con 0x7fe6cc0c4340 2023-10-30T12:57:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6cb7fe640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7fe6ac06ec80 0x7fe6ac071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c3fff640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6ac0664d0 0x7fe6ac068990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ad000 tx=0x7fe6b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6cb7fe640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7fe6ac06ec80 0x7fe6ac071120 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.798+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== osd.2 v2:172.21.15.8:6803/713612811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe6ac0717f0 con 0x7fe6ac06ec80 2023-10-30T12:57:43.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.810+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 --> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe6cc14cf40 con 0x7fe6ac06ec80 2023-10-30T12:57:43.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.810+0000 7fe6c8ff9640 1 -- 172.21.15.8:0/1883821768 <== osd.2 v2:172.21.15.8:6803/713612811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe6cc14cf40 con 0x7fe6ac06ec80 2023-10-30T12:57:43.818 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:43.818 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:43.818 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:43.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.810+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7fe6ac06ec80 msgr2=0x7fe6ac071120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.810+0000 7fe6d190f640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6803/713612811,v1:172.21.15.8:6808/713612811] conn(0x7fe6ac06ec80 0x7fe6ac071120 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6ac0664d0 msgr2=0x7fe6ac068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe6ac0664d0 0x7fe6ac068990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ad000 tx=0x7fe6b4008040 comp rx=0 tx=0).stop 2023-10-30T12:57:43.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 msgr2=0x7fe6cc156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 --2- 172.21.15.8:0/1883821768 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6cc0c4340 0x7fe6cc156b10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c3d0 tx=0x7fe6bc00c890 comp rx=0 tx=0).stop 2023-10-30T12:57:43.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6cb7fe640 1 -- 172.21.15.8:0/1883821768 reap_dead start 2023-10-30T12:57:43.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 shutdown_connections 2023-10-30T12:57:43.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 >> 172.21.15.8:0/1883821768 conn(0x7fe6cc0cf1c0 msgr2=0x7fe6cc0c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 shutdown_connections 2023-10-30T12:57:43.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.814+0000 7fe6d190f640 1 -- 172.21.15.8:0/1883821768 wait complete. 2023-10-30T12:57:43.836 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-30T12:57:43.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.974+0000 7fdac37b8640 1 Processor -- start 2023-10-30T12:57:43.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.974+0000 7fdac37b8640 1 -- start start 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdabc1537f0 0x7fdabc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdabc15a2e0 0x7fdabc0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc0cd4a0 0x7fdabc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0c3ab0 con 0x7fdabc15a2e0 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0706e0 con 0x7fdabc1537f0 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac37b8640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0c2aa0 con 0x7fdabc0cd4a0 2023-10-30T12:57:43.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdabc15a2e0 0x7fdabc0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdabc15a2e0 0x7fdabc0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48232/0 (socket says 172.21.15.8:48232) 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 -- 172.21.15.8:0/2328114308 learned_addr learned my addr 172.21.15.8:0/2328114308 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac1554640 1 --2- 172.21.15.8:0/2328114308 >> v2:172.21.15.46:3300/0 conn(0x7fdabc1537f0 0x7fdabc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac1d55640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc0cd4a0 0x7fdabc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 -- 172.21.15.8:0/2328114308 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc0cd4a0 msgr2=0x7fdabc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc0cd4a0 0x7fdabc06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 -- 172.21.15.8:0/2328114308 >> v2:172.21.15.46:3300/0 conn(0x7fdabc1537f0 msgr2=0x7fdabc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 --2- 172.21.15.8:0/2328114308 >> v2:172.21.15.46:3300/0 conn(0x7fdabc1537f0 0x7fdabc0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.978+0000 7fdac0d53640 1 -- 172.21.15.8:0/2328114308 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0708e0 con 0x7fdabc15a2e0 2023-10-30T12:57:43.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac0d53640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc15a2e0 0x7fdabc0ccf60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cdb70 tx=0x7fdaac01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d28564891f61c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdab3fff640 1 -- 172.21.15.8:0/2328114308 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0a89b0 con 0x7fdabc15a2e0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdab3fff640 1 -- 172.21.15.8:0/2328114308 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0a8b10 con 0x7fdabc15a2e0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdab3fff640 1 -- 172.21.15.8:0/2328114308 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0a8ea0 con 0x7fdabc15a2e0 2023-10-30T12:57:43.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- 172.21.15.8:0/2328114308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc15a2e0 msgr2=0x7fdabc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc15a2e0 0x7fdabc0ccf60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0cdb70 tx=0x7fdaac01ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- 172.21.15.8:0/2328114308 shutdown_connections 2023-10-30T12:57:43.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc0cd4a0 0x7fdabc06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- 172.21.15.8:0/2328114308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc15a2e0 0x7fdabc0ccf60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- 172.21.15.8:0/2328114308 >> v2:172.21.15.46:3300/0 conn(0x7fdabc1537f0 0x7fdabc0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- 172.21.15.8:0/2328114308 >> 172.21.15.8:0/2328114308 conn(0x7fdabc071fb0 msgr2=0x7fdabc0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:43.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- 172.21.15.8:0/2328114308 shutdown_connections 2023-10-30T12:57:43.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- 172.21.15.8:0/2328114308 wait complete. 2023-10-30T12:57:43.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 Processor -- start 2023-10-30T12:57:43.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- start start 2023-10-30T12:57:43.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc1537f0 0x7fdabc1f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdabc15a2e0 0x7fdabc1ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:43.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1efc40 con 0x7fdabc0cd4a0 2023-10-30T12:57:43.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1eff20 con 0x7fdabc15a2e0 2023-10-30T12:57:43.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.982+0000 7fdac37b8640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0c5b40 con 0x7fdabc1537f0 2023-10-30T12:57:43.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48234/0 (socket says 172.21.15.8:48234) 2023-10-30T12:57:43.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 -- 172.21.15.8:0/482608967 learned_addr learned my addr 172.21.15.8:0/482608967 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:43.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1d55640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdabc15a2e0 0x7fdabc1ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac0d53640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc1537f0 0x7fdabc1f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:43.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 -- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc1537f0 msgr2=0x7fdabc1f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc1537f0 0x7fdabc1f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 -- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdabc15a2e0 msgr2=0x7fdabc1ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:43.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdabc15a2e0 0x7fdabc1ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:43.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 -- 172.21.15.8:0/482608967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac0094d0 con 0x7fdabc0cd4a0 2023-10-30T12:57:43.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac1554640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d830 tx=0x7fdab800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:43.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800b6c0 con 0x7fdabc0cd4a0 2023-10-30T12:57:43.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab800b820 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc0c5de0 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800bb90 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc0c6380 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc04e170 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.986+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdab801e460 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.990+0000 7fdab1ffb640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 0x7fdaa00689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.990+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fdab80848a0 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.990+0000 7fdac0d53640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 0x7fdaa00689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.990+0000 7fdac0d53640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 0x7fdaa00689b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b2000 tx=0x7fdaac0aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:43.994+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdab8051af0 con 0x7fdabc0cd4a0 2023-10-30T12:57:44.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.150+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdabc0c2170 con 0x7fdaa00664f0 2023-10-30T12:57:44.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.150+0000 7fdab1ffb640 1 -- 172.21.15.8:0/482608967 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdaac0bc450 con 0x7fdaa00664f0 2023-10-30T12:57:44.159 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 2 to deep-scrub 2023-10-30T12:57:44.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 msgr2=0x7fdaa00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 0x7fdaa00689b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b2000 tx=0x7fdaac0aa070 comp rx=0 tx=0).stop 2023-10-30T12:57:44.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 msgr2=0x7fdabc1f03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d830 tx=0x7fdab800dcf0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 shutdown_connections 2023-10-30T12:57:44.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdaa00664f0 0x7fdaa00689b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdabc15a2e0 0x7fdabc1ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdabc1537f0 0x7fdabc1f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 --2- 172.21.15.8:0/482608967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdabc0cd4a0 0x7fdabc1f03b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 >> 172.21.15.8:0/482608967 conn(0x7fdabc071fb0 msgr2=0x7fdabc14d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 shutdown_connections 2023-10-30T12:57:44.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.154+0000 7fdac37b8640 1 -- 172.21.15.8:0/482608967 wait complete. 2023-10-30T12:57:44.183 INFO:tasks.ceph:Scrubbing osd.3 2023-10-30T12:57:44.183 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 Processor -- start 2023-10-30T12:57:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 -- start start 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f86681567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8668159330 con 0x7f86680c5be0 2023-10-30T12:57:44.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8668159bd0 con 0x7f86680c7910 2023-10-30T12:57:44.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866e8ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86680c07b0 con 0x7f866814c650 2023-10-30T12:57:44.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866ce87640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f86680c5be0 0x7f86681606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37996/0 (socket says 172.21.15.8:37996) 2023-10-30T12:57:44.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f866ce87640 1 -- 172.21.15.8:0/1676426049 learned_addr learned my addr 172.21.15.8:0/1676426049 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:44.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8667fff640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f86681567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954435540 0 0) 0x7f8668159330 con 0x7f86680c5be0 2023-10-30T12:57:44.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8650003450 con 0x7f86680c5be0 2023-10-30T12:57:44.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139214077 0 0) 0x7f8668159bd0 con 0x7f86680c7910 2023-10-30T12:57:44.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8668159330 con 0x7f86680c7910 2023-10-30T12:57:44.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666387504 0 0) 0x7f8650003450 con 0x7f86680c5be0 2023-10-30T12:57:44.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 msgr2=0x7f86681567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f86681567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 >> v1:172.21.15.46:6789/0 conn(0x7f86680c7910 legacy=0x7f86680c00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680c7e50 con 0x7f86680c5be0 2023-10-30T12:57:44.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f8660001a00 con 0x7f86680c5be0 2023-10-30T12:57:44.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 >> v1:172.21.15.8:6789/0 conn(0x7f86680c5be0 legacy=0x7f86681606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8650003c30 0x7f8650008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8650004110 0x7f865000adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f865000d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86680c07b0 con 0x7f8650004110 2023-10-30T12:57:44.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865000dc10 con 0x7f8650003c30 2023-10-30T12:57:44.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865000ddd0 con 0x7f866814c650 2023-10-30T12:57:44.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.326+0000 7f86677fe640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8650004110 0x7f865000adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f8667fff640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8650003c30 0x7f8650008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f866ce87640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f865000d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 -- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 msgr2=0x7f865000d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f866814c650 0x7f865000d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 -- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8650003c30 msgr2=0x7f8650008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8650003c30 0x7f8650008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 -- 172.21.15.8:0/1676426049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650003450 con 0x7f8650004110 2023-10-30T12:57:44.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f86677fe640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8650004110 0x7f865000adf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f865c0018d0 tx=0x7f865c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f56fa15113fc7d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0052d0 con 0x7f8650004110 2023-10-30T12:57:44.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c005450 con 0x7f8650004110 2023-10-30T12:57:44.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f8666ffd640 1 -- 172.21.15.8:0/1676426049 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0a8930 con 0x7f8650004110 2023-10-30T12:57:44.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f866e8ea640 1 -- 172.21.15.8:0/1676426049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8650004110 msgr2=0x7f865000adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.347 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f866e8ea640 1 --2- 172.21.15.8:0/1676426049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8650004110 0x7f865000adf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f865c0018d0 tx=0x7f865c01ecd0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.330+0000 7f866ce87640 1 -- 172.21.15.8:0/1676426049 reap_dead start 2023-10-30T12:57:44.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- 172.21.15.8:0/1676426049 shutdown_connections 2023-10-30T12:57:44.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- 172.21.15.8:0/1676426049 >> 172.21.15.8:0/1676426049 conn(0x7f866806e150 msgr2=0x7f8668069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- 172.21.15.8:0/1676426049 shutdown_connections 2023-10-30T12:57:44.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- 172.21.15.8:0/1676426049 wait complete. 2023-10-30T12:57:44.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 Processor -- start 2023-10-30T12:57:44.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- start start 2023-10-30T12:57:44.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8650004110 0x7f86681f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8668072750 0x7f86680c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 0x7f86680c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681591a0 con 0x7f86681efa40 2023-10-30T12:57:44.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681efec0 con 0x7f8668072750 2023-10-30T12:57:44.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866e8ea640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8668148560 con 0x7f8650004110 2023-10-30T12:57:44.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 0x7f86680c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 0x7f86680c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48240/0 (socket says 172.21.15.8:48240) 2023-10-30T12:57:44.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 -- 172.21.15.8:0/2818179407 learned_addr learned my addr 172.21.15.8:0/2818179407 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:44.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f86677fe640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8668072750 0x7f86680c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f8667fff640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8650004110 0x7f86681f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 -- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8650004110 msgr2=0x7f86681f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8650004110 0x7f86681f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 -- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8668072750 msgr2=0x7f86680c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8668072750 0x7f86680c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86681ef5b0 con 0x7f86681efa40 2023-10-30T12:57:44.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f866ce87640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 0x7f86680c9400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86680c78e0 tx=0x7f866000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866000eb70 con 0x7f86681efa40 2023-10-30T12:57:44.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.334+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866000ecd0 con 0x7f86681efa40 2023-10-30T12:57:44.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86681487b0 con 0x7f86681efa40 2023-10-30T12:57:44.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660014430 con 0x7f86681efa40 2023-10-30T12:57:44.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86681ef240 con 0x7f86681efa40 2023-10-30T12:57:44.355 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86680c80f0 con 0x7f86681efa40 2023-10-30T12:57:44.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8660014590 con 0x7f86681efa40 2023-10-30T12:57:44.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86540664a0 0x7f8654068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f8660083f60 con 0x7f86681efa40 2023-10-30T12:57:44.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8667fff640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86540664a0 0x7f8654068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7f865406ec50 0x7f86540710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86540717c0 con 0x7f865406ec50 2023-10-30T12:57:44.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8660086540 con 0x7f86681efa40 2023-10-30T12:57:44.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f86677fe640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7f865406ec50 0x7f86540710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f8667fff640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86540664a0 0x7f8654068960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8658009950 tx=0x7f8658008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.338+0000 7f86677fe640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7f865406ec50 0x7f86540710f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.342+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== osd.3 v2:172.21.15.8:6800/469140225 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f86540717c0 con 0x7f865406ec50 2023-10-30T12:57:44.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 --> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86680c6a10 con 0x7f865406ec50 2023-10-30T12:57:44.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f8664ff9640 1 -- 172.21.15.8:0/2818179407 <== osd.3 v2:172.21.15.8:6800/469140225 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f865c0052b0 con 0x7f865406ec50 2023-10-30T12:57:44.361 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:44.361 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:44.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7f865406ec50 msgr2=0x7f86540710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6800/469140225,v1:172.21.15.8:6802/469140225] conn(0x7f865406ec50 0x7f86540710f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86540664a0 msgr2=0x7f8654068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f86540664a0 0x7f8654068960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8658009950 tx=0x7f8658008040 comp rx=0 tx=0).stop 2023-10-30T12:57:44.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 msgr2=0x7f86680c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 --2- 172.21.15.8:0/2818179407 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f86681efa40 0x7f86680c9400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86680c78e0 tx=0x7f866000e160 comp rx=0 tx=0).stop 2023-10-30T12:57:44.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866ce87640 1 -- 172.21.15.8:0/2818179407 reap_dead start 2023-10-30T12:57:44.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 shutdown_connections 2023-10-30T12:57:44.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 >> 172.21.15.8:0/2818179407 conn(0x7f866806e150 msgr2=0x7f866814b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 shutdown_connections 2023-10-30T12:57:44.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.354+0000 7f866e8ea640 1 -- 172.21.15.8:0/2818179407 wait complete. 2023-10-30T12:57:44.378 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-30T12:57:44.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 Processor -- start 2023-10-30T12:57:44.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 -- start start 2023-10-30T12:57:44.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7730151840 0x7f77300c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.524 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300c5030 con 0x7f7730151c20 2023-10-30T12:57:44.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7730157830 con 0x7f77300cb8b0 2023-10-30T12:57:44.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f7735fd3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7730157b10 con 0x7f7730151840 2023-10-30T12:57:44.525 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772effd640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7730151c20 0x7f7730072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:37998/0 (socket says 172.21.15.8:37998) 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772effd640 1 -- 172.21.15.8:0/1439249445 learned_addr learned my addr 172.21.15.8:0/1439249445 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772f7fe640 1 --2- 172.21.15.8:0/1439249445 >> v2:172.21.15.46:3300/0 conn(0x7f7730151840 0x7f77300c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716659689 0 0) 0x7f77300c5030 con 0x7f7730151c20 2023-10-30T12:57:44.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7718003530 con 0x7f7730151c20 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358400153 0 0) 0x7f7718003530 con 0x7f7730151c20 2023-10-30T12:57:44.528 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 >> v2:172.21.15.46:3300/0 conn(0x7f7730151840 msgr2=0x7f77300c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 --2- 172.21.15.8:0/1439249445 >> v2:172.21.15.46:3300/0 conn(0x7f7730151840 0x7f77300c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 >> v1:172.21.15.46:6789/0 conn(0x7f77300cb8b0 legacy=0x7f773006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.529 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7730157df0 con 0x7f7730151c20 2023-10-30T12:57:44.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f7724001680 con 0x7f7730151c20 2023-10-30T12:57:44.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 >> v1:172.21.15.8:6789/0 conn(0x7f7730151c20 legacy=0x7f7730072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7718003980 0x7f7718003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.530 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77180047d0 0x7f7718004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f771800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730157b10 con 0x7f7718005190 2023-10-30T12:57:44.531 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771800c040 con 0x7f7718003980 2023-10-30T12:57:44.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771800c200 con 0x7f77180047d0 2023-10-30T12:57:44.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772ffff640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f771800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772f7fe640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7718003980 0x7f7718003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.532 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.518+0000 7f772effd640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77180047d0 0x7f7718004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 -- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77180047d0 msgr2=0x7f7718004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77180047d0 0x7f7718004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 -- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7718003980 msgr2=0x7f7718003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7718003980 0x7f7718003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 -- 172.21.15.8:0/1439249445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718003530 con 0x7f7718005190 2023-10-30T12:57:44.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772effd640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f77180047d0 0x7f7718004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:44.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 -- 172.21.15.8:0/1439249445 reap_dead start 2023-10-30T12:57:44.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772ffff640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f771800b990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7728001e70 tx=0x7f772801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f914c85122876db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003b80 con 0x7f7718005190 2023-10-30T12:57:44.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728003d20 con 0x7f7718005190 2023-10-30T12:57:44.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f772e7fc640 1 -- 172.21.15.8:0/1439249445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728005260 con 0x7f7718005190 2023-10-30T12:57:44.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 -- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 msgr2=0x7f771800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f771800b990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7728001e70 tx=0x7f772801c760 comp rx=0 tx=0).stop 2023-10-30T12:57:44.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 -- 172.21.15.8:0/1439249445 shutdown_connections 2023-10-30T12:57:44.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 --2- 172.21.15.8:0/1439249445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f771800b990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 -- 172.21.15.8:0/1439249445 >> 172.21.15.8:0/1439249445 conn(0x7f77300cac70 msgr2=0x7f77300cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 -- 172.21.15.8:0/1439249445 shutdown_connections 2023-10-30T12:57:44.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.522+0000 7f7735fd3640 1 -- 172.21.15.8:0/1439249445 wait complete. 2023-10-30T12:57:44.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 Processor -- start 2023-10-30T12:57:44.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 -- start start 2023-10-30T12:57:44.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7730070950 0x7f77301ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300c8a50 0x7f77300720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301e1280 con 0x7f7718005190 2023-10-30T12:57:44.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77301eede0 con 0x7f77300c8a50 2023-10-30T12:57:44.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77301e14f0 con 0x7f7730070950 2023-10-30T12:57:44.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48264/0 (socket says 172.21.15.8:48264) 2023-10-30T12:57:44.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 -- 172.21.15.8:0/2307719916 learned_addr learned my addr 172.21.15.8:0/2307719916 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:44.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772effd640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7730070950 0x7f77301ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772ffff640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300c8a50 0x7f77300720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 -- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7730070950 msgr2=0x7f77301ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7730070950 0x7f77301ef8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 -- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300c8a50 msgr2=0x7f77300720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300c8a50 0x7f77300720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 -- 172.21.15.8:0/2307719916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77301e17d0 con 0x7f7718005190 2023-10-30T12:57:44.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f772f7fe640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7720011960 tx=0x7f7720011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000faf0 con 0x7f7718005190 2023-10-30T12:57:44.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000fc50 con 0x7f7718005190 2023-10-30T12:57:44.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.526+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77301ef0a0 con 0x7f7718005190 2023-10-30T12:57:44.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720014440 con 0x7f7718005190 2023-10-30T12:57:44.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7730072630 con 0x7f7718005190 2023-10-30T12:57:44.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773004e170 con 0x7f7718005190 2023-10-30T12:57:44.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f77200145a0 con 0x7f7718005190 2023-10-30T12:57:44.547 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7717fff640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 0x7f771c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f77200849f0 con 0x7f7718005190 2023-10-30T12:57:44.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f772effd640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 0x7f771c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.530+0000 7f772effd640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 0x7f771c068990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77301e1b90 tx=0x7f7724000ee0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.534+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7720051b30 con 0x7f7718005190 2023-10-30T12:57:44.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.694+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f77301580d0 con 0x7f771c0664d0 2023-10-30T12:57:44.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.694+0000 7f7717fff640 1 -- 172.21.15.8:0/2307719916 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7724005d50 con 0x7f771c0664d0 2023-10-30T12:57:44.701 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 3 to deep-scrub 2023-10-30T12:57:44.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 msgr2=0x7f771c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 0x7f771c068990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77301e1b90 tx=0x7f7724000ee0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 msgr2=0x7f77301dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7720011960 tx=0x7f7720011e20 comp rx=0 tx=0).stop 2023-10-30T12:57:44.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 shutdown_connections 2023-10-30T12:57:44.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f771c0664d0 0x7f771c068990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300c8a50 0x7f77300720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7730070950 0x7f77301ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 --2- 172.21.15.8:0/2307719916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7718005190 0x7f77301dfb30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 >> 172.21.15.8:0/2307719916 conn(0x7f77300cac70 msgr2=0x7f773014ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 shutdown_connections 2023-10-30T12:57:44.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.698+0000 7f7735fd3640 1 -- 172.21.15.8:0/2307719916 wait complete. 2023-10-30T12:57:44.722 INFO:tasks.ceph:Scrubbing osd.4 2023-10-30T12:57:44.722 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:44.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 Processor -- start 2023-10-30T12:57:44.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 -- start start 2023-10-30T12:57:44.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9ab41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab406af00 con 0x7f9ab40c4340 2023-10-30T12:57:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab406a660 con 0x7f9ab40ca470 2023-10-30T12:57:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9abb06b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c4cd0 con 0x7f9ab40c4720 2023-10-30T12:57:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab8e07640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f9ab40c4340 0x7f9ab4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38000/0 (socket says 172.21.15.8:38000) 2023-10-30T12:57:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab8e07640 1 -- 172.21.15.8:0/2436819262 learned_addr learned my addr 172.21.15.8:0/2436819262 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9ab41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172381464 0 0) 0x7f9ab406af00 con 0x7f9ab40c4340 2023-10-30T12:57:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa4003530 con 0x7f9ab40c4340 2023-10-30T12:57:44.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666382051 0 0) 0x7f9ab406a660 con 0x7f9ab40ca470 2023-10-30T12:57:44.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab406af00 con 0x7f9ab40ca470 2023-10-30T12:57:44.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397557606 0 0) 0x7f9aa4003530 con 0x7f9ab40c4340 2023-10-30T12:57:44.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 msgr2=0x7f9ab41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9ab41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 >> v1:172.21.15.46:6789/0 conn(0x7f9ab40ca470 legacy=0x7f9ab4153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab406a940 con 0x7f9ab40c4340 2023-10-30T12:57:44.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f9aa80016a0 con 0x7f9ab40c4340 2023-10-30T12:57:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 >> v1:172.21.15.8:6789/0 conn(0x7f9ab40c4340 legacy=0x7f9ab4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9aa4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa40040a0 0x7f9aa40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aa4004d10 0x7f9aa4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c4cd0 con 0x7f9aa4004d10 2023-10-30T12:57:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa400bb20 con 0x7f9aa40040a0 2023-10-30T12:57:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa400bcb0 con 0x7f9ab40c4720 2023-10-30T12:57:44.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab9608640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aa4004d10 0x7f9aa4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab8e07640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9aa4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.866+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa40040a0 0x7f9aa40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 -- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 msgr2=0x7f9aa4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9ab40c4720 0x7f9aa4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 -- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa40040a0 msgr2=0x7f9aa40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:44.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aa40040a0 0x7f9aa40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 -- 172.21.15.8:0/2436819262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4003530 con 0x7f9aa4004d10 2023-10-30T12:57:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aa4004d10 0x7f9aa4005190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9aac001fd0 tx=0x7f9aac01f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85054603265c19db server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0a8c70 con 0x7f9aa4004d10 2023-10-30T12:57:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac0a8df0 con 0x7f9aa4004d10 2023-10-30T12:57:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab37fe640 1 -- 172.21.15.8:0/2436819262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac003bc0 con 0x7f9aa4004d10 2023-10-30T12:57:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9abb06b640 1 -- 172.21.15.8:0/2436819262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aa4004d10 msgr2=0x7f9aa4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9abb06b640 1 --2- 172.21.15.8:0/2436819262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aa4004d10 0x7f9aa4005190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9aac001fd0 tx=0x7f9aac01f050 comp rx=0 tx=0).stop 2023-10-30T12:57:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.870+0000 7f9ab9608640 1 -- 172.21.15.8:0/2436819262 reap_dead start 2023-10-30T12:57:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- 172.21.15.8:0/2436819262 shutdown_connections 2023-10-30T12:57:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- 172.21.15.8:0/2436819262 >> 172.21.15.8:0/2436819262 conn(0x7f9ab40cf1c0 msgr2=0x7f9ab40cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- 172.21.15.8:0/2436819262 shutdown_connections 2023-10-30T12:57:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- 172.21.15.8:0/2436819262 wait complete. 2023-10-30T12:57:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 Processor -- start 2023-10-30T12:57:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- start start 2023-10-30T12:57:44.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9aa4004d10 0x7f9ab40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab40b2040 0x7f9ab4155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 0x7f9ab414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab41556e0 con 0x7f9ab41560b0 2023-10-30T12:57:44.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4156ac0 con 0x7f9ab40b2040 2023-10-30T12:57:44.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9abb06b640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4156530 con 0x7f9aa4004d10 2023-10-30T12:57:44.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 0x7f9ab414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 0x7f9ab414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48280/0 (socket says 172.21.15.8:48280) 2023-10-30T12:57:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 -- 172.21.15.8:0/703853065 learned_addr learned my addr 172.21.15.8:0/703853065 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab8e07640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9aa4004d10 0x7f9ab40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab40b2040 0x7f9ab4155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 -- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9aa4004d10 msgr2=0x7f9ab40c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f9aa4004d10 0x7f9ab40c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 -- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab40b2040 msgr2=0x7f9ab4155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab40b2040 0x7f9ab4155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4156820 con 0x7f9ab41560b0 2023-10-30T12:57:44.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab40b2040 0x7f9ab4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-30T12:57:44.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab9608640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 0x7f9ab414f840 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9aac001fd0 tx=0x7f9aa4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.874+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac003e60 con 0x7f9ab41560b0 2023-10-30T12:57:44.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac0a8a70 con 0x7f9ab41560b0 2023-10-30T12:57:44.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab41ef280 con 0x7f9ab41560b0 2023-10-30T12:57:44.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac006b60 con 0x7f9ab41560b0 2023-10-30T12:57:44.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab4155090 con 0x7f9ab41560b0 2023-10-30T12:57:44.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ab40c9c70 con 0x7f9ab41560b0 2023-10-30T12:57:44.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9aac006cc0 con 0x7f9ab41560b0 2023-10-30T12:57:44.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9ab17fa640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9a980664a0 0x7f9a98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.878+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f9aac1214d0 con 0x7f9ab41560b0 2023-10-30T12:57:44.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab17fa640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f9a9806ec50 0x7f9a980710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:44.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a980717c0 con 0x7f9a9806ec50 2023-10-30T12:57:44.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aac121840 con 0x7f9ab41560b0 2023-10-30T12:57:44.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab8e07640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9a980664a0 0x7f9a98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f9a9806ec50 0x7f9a980710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:44.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab8e07640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9a980664a0 0x7f9a98068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8001210 tx=0x7f9aa8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab3fff640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f9a9806ec50 0x7f9a980710f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:44.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.882+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== osd.4 v2:172.21.15.46:6810/4276465018 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f9a980717c0 con 0x7f9a9806ec50 2023-10-30T12:57:44.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 --> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9ab406ac20 con 0x7f9a9806ec50 2023-10-30T12:57:44.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9ab17fa640 1 -- 172.21.15.8:0/703853065 <== osd.4 v2:172.21.15.46:6810/4276465018 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9aa0009bb0 con 0x7f9a9806ec50 2023-10-30T12:57:44.904 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:44.904 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:44.905 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:44.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f9a9806ec50 msgr2=0x7f9a980710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.46:6810/4276465018,v1:172.21.15.46:6812/4276465018] conn(0x7f9a9806ec50 0x7f9a980710f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:44.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9a980664a0 msgr2=0x7f9a98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f9a980664a0 0x7f9a98068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8001210 tx=0x7f9aa8009040 comp rx=0 tx=0).stop 2023-10-30T12:57:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 msgr2=0x7f9ab414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 --2- 172.21.15.8:0/703853065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ab41560b0 0x7f9ab414f840 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9aac001fd0 tx=0x7f9aa4005800 comp rx=0 tx=0).stop 2023-10-30T12:57:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9ab9608640 1 -- 172.21.15.8:0/703853065 reap_dead start 2023-10-30T12:57:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 shutdown_connections 2023-10-30T12:57:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 >> 172.21.15.8:0/703853065 conn(0x7f9ab40cf1c0 msgr2=0x7f9ab40cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 shutdown_connections 2023-10-30T12:57:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:44.898+0000 7f9abb06b640 1 -- 172.21.15.8:0/703853065 wait complete. 2023-10-30T12:57:44.923 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-30T12:57:45.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 Processor -- start 2023-10-30T12:57:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 -- start start 2023-10-30T12:57:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 0x7fe36015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe3600c42a0 0x7fe3600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe360159f20 con 0x7fe3600c4a10 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36015dec0 con 0x7fe3600c42a0 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe367f72640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3601599a0 con 0x7fe360153c80 2023-10-30T12:57:45.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36550d640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe3600c4a10 0x7fe36015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38010/0 (socket says 172.21.15.8:38010) 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36550d640 1 -- 172.21.15.8:0/1598172171 learned_addr learned my addr 172.21.15.8:0/1598172171 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36650f640 1 --2- 172.21.15.8:0/1598172171 >> v2:172.21.15.8:3300/0 conn(0x7fe3600c42a0 0x7fe3600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe365d0e640 1 --2- 172.21.15.8:0/1598172171 >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 0x7fe36015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe364d0c640 1 -- 172.21.15.8:0/1598172171 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982161292 0 0) 0x7fe360159f20 con 0x7fe3600c4a10 2023-10-30T12:57:45.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe364d0c640 1 -- 172.21.15.8:0/1598172171 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34c003530 con 0x7fe3600c4a10 2023-10-30T12:57:45.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36650f640 1 -- 172.21.15.8:0/1598172171 >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 msgr2=0x7fe36015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:45.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36650f640 1 --2- 172.21.15.8:0/1598172171 >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 0x7fe36015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36650f640 1 -- 172.21.15.8:0/1598172171 >> v1:172.21.15.8:6789/0 conn(0x7fe3600c4a10 legacy=0x7fe36015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe36650f640 1 -- 172.21.15.8:0/1598172171 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36015d620 con 0x7fe3600c42a0 2023-10-30T12:57:45.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.062+0000 7fe365d0e640 1 --2- 172.21.15.8:0/1598172171 >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 0x7fe36015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe36650f640 1 --2- 172.21.15.8:0/1598172171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3600c42a0 0x7fe3600ca1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d00f0 tx=0x7fe35801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84fbe21155aa2c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe364d0c640 1 -- 172.21.15.8:0/1598172171 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580b0440 con 0x7fe3600c42a0 2023-10-30T12:57:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe364d0c640 1 -- 172.21.15.8:0/1598172171 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580b05a0 con 0x7fe3600c42a0 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe364d0c640 1 -- 172.21.15.8:0/1598172171 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580b0930 con 0x7fe3600c42a0 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 -- 172.21.15.8:0/1598172171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3600c42a0 msgr2=0x7fe3600ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 --2- 172.21.15.8:0/1598172171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3600c42a0 0x7fe3600ca1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d00f0 tx=0x7fe35801f0f0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 -- 172.21.15.8:0/1598172171 shutdown_connections 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 --2- 172.21.15.8:0/1598172171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3600c42a0 0x7fe3600ca1e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 --2- 172.21.15.8:0/1598172171 >> v2:172.21.15.46:3300/0 conn(0x7fe360153c80 0x7fe36015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 -- 172.21.15.8:0/1598172171 >> 172.21.15.8:0/1598172171 conn(0x7fe3600715c0 msgr2=0x7fe3600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 -- 172.21.15.8:0/1598172171 shutdown_connections 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.066+0000 7fe367f72640 1 -- 172.21.15.8:0/1598172171 wait complete. 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 Processor -- start 2023-10-30T12:57:45.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- start start 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3600c42a0 0x7fe36014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe3600c4a10 0x7fe36014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600cfe10 con 0x7fe360153c80 2023-10-30T12:57:45.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36014fc50 con 0x7fe3600c42a0 2023-10-30T12:57:45.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36014ff30 con 0x7fe3600c4a10 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48288/0 (socket says 172.21.15.8:48288) 2023-10-30T12:57:45.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 -- 172.21.15.8:0/1364806429 learned_addr learned my addr 172.21.15.8:0/1364806429 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe365d0e640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3600c42a0 0x7fe36014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36550d640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe3600c4a10 0x7fe36014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 -- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe3600c4a10 msgr2=0x7fe36014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe3600c4a10 0x7fe36014d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 -- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3600c42a0 msgr2=0x7fe36014d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3600c42a0 0x7fe36014d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 -- 172.21.15.8:0/1364806429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358009a70 con 0x7fe360153c80 2023-10-30T12:57:45.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe36650f640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe3580b3040 tx=0x7fe35801be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580b0440 con 0x7fe360153c80 2023-10-30T12:57:45.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580ba820 con 0x7fe360153c80 2023-10-30T12:57:45.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36014deb0 con 0x7fe360153c80 2023-10-30T12:57:45.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580bac10 con 0x7fe360153c80 2023-10-30T12:57:45.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.070+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe360150370 con 0x7fe360153c80 2023-10-30T12:57:45.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36014dd10 con 0x7fe360153c80 2023-10-30T12:57:45.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3580b7050 con 0x7fe360153c80 2023-10-30T12:57:45.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe3567fc640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fe3581227e0 con 0x7fe360153c80 2023-10-30T12:57:45.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe365d0e640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.074+0000 7fe365d0e640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 0x7fe344068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00ba10 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.078+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe36014dd10 con 0x7fe360153c80 2023-10-30T12:57:45.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe36015dbe0 con 0x7fe3440664a0 2023-10-30T12:57:45.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe3567fc640 1 -- 172.21.15.8:0/1364806429 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe35c004020 con 0x7fe3440664a0 2023-10-30T12:57:45.244 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 4 to deep-scrub 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 msgr2=0x7fe344068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 0x7fe344068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00ba10 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2023-10-30T12:57:45.245 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 msgr2=0x7fe36014e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe3580b3040 tx=0x7fe35801be90 comp rx=0 tx=0).stop 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 shutdown_connections 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe360153c80 0x7fe36014e6b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe3600c4a10 0x7fe36014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 --2- 172.21.15.8:0/1364806429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3600c42a0 0x7fe36014d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 >> 172.21.15.8:0/1364806429 conn(0x7fe3600715c0 msgr2=0x7fe360158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 shutdown_connections 2023-10-30T12:57:45.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.238+0000 7fe367f72640 1 -- 172.21.15.8:0/1364806429 wait complete. 2023-10-30T12:57:45.265 INFO:tasks.ceph:Scrubbing osd.5 2023-10-30T12:57:45.265 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:45.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 Processor -- start 2023-10-30T12:57:45.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 -- start start 2023-10-30T12:57:45.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc7006af10 0x7fdc7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fdc700c9910 0x7fdc700cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc700ca7c0 con 0x7fdc70153920 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700cad80 con 0x7fdc700c9910 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc76103640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700caaa0 con 0x7fdc7006af10 2023-10-30T12:57:45.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75101640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fdc70153920 0x7fdc7006a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38024/0 (socket says 172.21.15.8:38024) 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75101640 1 -- 172.21.15.8:0/870970795 learned_addr learned my addr 172.21.15.8:0/870970795 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75902640 1 --2- 172.21.15.8:0/870970795 >> v2:172.21.15.8:3300/0 conn(0x7fdc700c9910 0x7fdc700cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc74900640 1 --2- 172.21.15.8:0/870970795 >> v2:172.21.15.46:3300/0 conn(0x7fdc7006af10 0x7fdc7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc67fff640 1 -- 172.21.15.8:0/870970795 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316782002 0 0) 0x7fdc700ca7c0 con 0x7fdc70153920 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc67fff640 1 -- 172.21.15.8:0/870970795 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c003530 con 0x7fdc70153920 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75902640 1 -- 172.21.15.8:0/870970795 >> v2:172.21.15.46:3300/0 conn(0x7fdc7006af10 msgr2=0x7fdc7006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:45.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75902640 1 --2- 172.21.15.8:0/870970795 >> v2:172.21.15.46:3300/0 conn(0x7fdc7006af10 0x7fdc7006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75902640 1 -- 172.21.15.8:0/870970795 >> v1:172.21.15.8:6789/0 conn(0x7fdc70153920 legacy=0x7fdc7006a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.414+0000 7fdc75902640 1 -- 172.21.15.8:0/870970795 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7015a2c0 con 0x7fdc700c9910 2023-10-30T12:57:45.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc75902640 1 --2- 172.21.15.8:0/870970795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc700cb460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdc700b1e30 tx=0x7fdc6801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae241a76d020aec server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc67fff640 1 -- 172.21.15.8:0/870970795 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680b0440 con 0x7fdc700c9910 2023-10-30T12:57:45.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc67fff640 1 -- 172.21.15.8:0/870970795 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc680b05a0 con 0x7fdc700c9910 2023-10-30T12:57:45.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc67fff640 1 -- 172.21.15.8:0/870970795 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680b0930 con 0x7fdc700c9910 2023-10-30T12:57:45.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 -- 172.21.15.8:0/870970795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 msgr2=0x7fdc700cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 --2- 172.21.15.8:0/870970795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc700cb460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdc700b1e30 tx=0x7fdc6801f0f0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 -- 172.21.15.8:0/870970795 shutdown_connections 2023-10-30T12:57:45.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 --2- 172.21.15.8:0/870970795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc700cb460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 --2- 172.21.15.8:0/870970795 >> v2:172.21.15.46:3300/0 conn(0x7fdc7006af10 0x7fdc7006c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 -- 172.21.15.8:0/870970795 >> 172.21.15.8:0/870970795 conn(0x7fdc700cf6d0 msgr2=0x7fdc7006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 -- 172.21.15.8:0/870970795 shutdown_connections 2023-10-30T12:57:45.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.418+0000 7fdc76103640 1 -- 172.21.15.8:0/870970795 wait complete. 2023-10-30T12:57:45.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 Processor -- start 2023-10-30T12:57:45.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- start start 2023-10-30T12:57:45.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdc7006af10 0x7fdc701498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc70149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70153920 0x7fdc7014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7014d070 con 0x7fdc700c9910 2023-10-30T12:57:45.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701ef170 con 0x7fdc70153920 2023-10-30T12:57:45.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701ef450 con 0x7fdc7006af10 2023-10-30T12:57:45.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc70149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc70149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48314/0 (socket says 172.21.15.8:48314) 2023-10-30T12:57:45.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 -- 172.21.15.8:0/4275980107 learned_addr learned my addr 172.21.15.8:0/4275980107 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:45.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc75101640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdc7006af10 0x7fdc701498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc75902640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70153920 0x7fdc7014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 -- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdc7006af10 msgr2=0x7fdc701498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdc7006af10 0x7fdc701498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 -- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70153920 msgr2=0x7fdc7014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70153920 0x7fdc7014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68009a70 con 0x7fdc700c9910 2023-10-30T12:57:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc75101640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fdc7006af10 0x7fdc701498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc74900640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc70149e10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e930 tx=0x7fdc6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60009690 con 0x7fdc700c9910 2023-10-30T12:57:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000ca00 con 0x7fdc700c9910 2023-10-30T12:57:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000cd70 con 0x7fdc700c9910 2023-10-30T12:57:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc701ef730 con 0x7fdc700c9910 2023-10-30T12:57:45.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.422+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc701efcf0 con 0x7fdc700c9910 2023-10-30T12:57:45.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc7006a2e0 con 0x7fdc700c9910 2023-10-30T12:57:45.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc60013470 con 0x7fdc700c9910 2023-10-30T12:57:45.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdc540664a0 0x7fdc54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fdc600842b0 con 0x7fdc700c9910 2023-10-30T12:57:45.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc75101640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdc540664a0 0x7fdc54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fdc5406ec50 0x7fdc540710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc540717c0 con 0x7fdc5406ec50 2023-10-30T12:57:45.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc60086890 con 0x7fdc700c9910 2023-10-30T12:57:45.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc75902640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fdc5406ec50 0x7fdc540710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc75101640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdc540664a0 0x7fdc54068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0011f0 tx=0x7fdc6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.426+0000 7fdc75902640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fdc5406ec50 0x7fdc540710f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.430+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== osd.5 v2:172.21.15.46:6800/3519076799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fdc540717c0 con 0x7fdc5406ec50 2023-10-30T12:57:45.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 --> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdc7015a5a0 con 0x7fdc5406ec50 2023-10-30T12:57:45.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc65ffb640 1 -- 172.21.15.8:0/4275980107 <== osd.5 v2:172.21.15.46:6800/3519076799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdc680b08d0 con 0x7fdc5406ec50 2023-10-30T12:57:45.448 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:45.449 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:45.449 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:45.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fdc5406ec50 msgr2=0x7fdc540710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.46:6800/3519076799,v1:172.21.15.46:6801/3519076799] conn(0x7fdc5406ec50 0x7fdc540710f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdc540664a0 msgr2=0x7fdc54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fdc540664a0 0x7fdc54068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0011f0 tx=0x7fdc6c009040 comp rx=0 tx=0).stop 2023-10-30T12:57:45.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 msgr2=0x7fdc70149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 --2- 172.21.15.8:0/4275980107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdc700c9910 0x7fdc70149e10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e930 tx=0x7fdc6000edf0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc75902640 1 -- 172.21.15.8:0/4275980107 reap_dead start 2023-10-30T12:57:45.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 shutdown_connections 2023-10-30T12:57:45.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 >> 172.21.15.8:0/4275980107 conn(0x7fdc700cf6d0 msgr2=0x7fdc70151440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 shutdown_connections 2023-10-30T12:57:45.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.442+0000 7fdc76103640 1 -- 172.21.15.8:0/4275980107 wait complete. 2023-10-30T12:57:45.468 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-30T12:57:45.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.606+0000 7fed9aa2f640 1 Processor -- start 2023-10-30T12:57:45.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.606+0000 7fed9aa2f640 1 -- start start 2023-10-30T12:57:45.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9aa2f640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed940ca0d0 0x7fed941596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9aa2f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed940c5560 0x7fed94153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9aa2f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940c4910 con 0x7fed940ca4b0 2023-10-30T12:57:45.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9aa2f640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940b27b0 con 0x7fed940ca0d0 2023-10-30T12:57:45.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9aa2f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940c4b10 con 0x7fed940c5560 2023-10-30T12:57:45.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed940ca0d0 0x7fed941596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fed940ca0d0 0x7fed941596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48324/0 (socket says 172.21.15.8:48324) 2023-10-30T12:57:45.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 -- 172.21.15.8:0/758364835 learned_addr learned my addr 172.21.15.8:0/758364835 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:45.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9922c640 1 --1- 172.21.15.8:0/758364835 >> v1:172.21.15.8:6789/0 conn(0x7fed940ca4b0 0x7fed940c4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38038/0 (socket says 172.21.15.8:38038) 2023-10-30T12:57:45.619 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed9a22e640 1 --2- 172.21.15.8:0/758364835 >> v2:172.21.15.46:3300/0 conn(0x7fed940c5560 0x7fed94153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 -- 172.21.15.8:0/758364835 >> v2:172.21.15.46:3300/0 conn(0x7fed940c5560 msgr2=0x7fed94153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 --2- 172.21.15.8:0/758364835 >> v2:172.21.15.46:3300/0 conn(0x7fed940c5560 0x7fed94153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 -- 172.21.15.8:0/758364835 >> v1:172.21.15.8:6789/0 conn(0x7fed940ca4b0 legacy=0x7fed940c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.610+0000 7fed99a2d640 1 -- 172.21.15.8:0/758364835 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940c50d0 con 0x7fed940ca0d0 2023-10-30T12:57:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed99a2d640 1 --2- 172.21.15.8:0/758364835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed941596f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed940c5040 tx=0x7fed9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1ba9dadbea8ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed98a2b640 1 -- 172.21.15.8:0/758364835 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900aa430 con 0x7fed940ca0d0 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed98a2b640 1 -- 172.21.15.8:0/758364835 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed900aa590 con 0x7fed940ca0d0 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed98a2b640 1 -- 172.21.15.8:0/758364835 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900aa920 con 0x7fed940ca0d0 2023-10-30T12:57:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 -- 172.21.15.8:0/758364835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 msgr2=0x7fed941596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/758364835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed941596f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed940c5040 tx=0x7fed9001c4d0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 -- 172.21.15.8:0/758364835 shutdown_connections 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/758364835 >> v2:172.21.15.46:3300/0 conn(0x7fed940c5560 0x7fed94153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/758364835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed941596f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 -- 172.21.15.8:0/758364835 >> 172.21.15.8:0/758364835 conn(0x7fed9406bf20 msgr2=0x7fed9406c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 -- 172.21.15.8:0/758364835 shutdown_connections 2023-10-30T12:57:45.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.614+0000 7fed9aa2f640 1 -- 172.21.15.8:0/758364835 wait complete. 2023-10-30T12:57:45.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 Processor -- start 2023-10-30T12:57:45.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- start start 2023-10-30T12:57:45.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed940c5560 0x7fed9414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 0x7fed94147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cd770 con 0x7fed940ca0d0 2023-10-30T12:57:45.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed941f6470 con 0x7fed940ca4b0 2023-10-30T12:57:45.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed94148c30 con 0x7fed940c5560 2023-10-30T12:57:45.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48336/0 (socket says 172.21.15.8:48336) 2023-10-30T12:57:45.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 -- 172.21.15.8:0/3583482394 learned_addr learned my addr 172.21.15.8:0/3583482394 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:45.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9a22e640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 0x7fed94147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed99a2d640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed940c5560 0x7fed9414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 -- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed940c5560 msgr2=0x7fed9414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed940c5560 0x7fed9414f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 -- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 msgr2=0x7fed94147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 0x7fed94147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 -- 172.21.15.8:0/3583482394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed900094d0 con 0x7fed940ca0d0 2023-10-30T12:57:45.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9a22e640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 0x7fed94147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:45.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9922c640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c490 tx=0x7fed8400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84007480 con 0x7fed940ca0d0 2023-10-30T12:57:45.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840075e0 con 0x7fed940ca0d0 2023-10-30T12:57:45.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84004a10 con 0x7fed940ca0d0 2023-10-30T12:57:45.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed940cd490 con 0x7fed940ca0d0 2023-10-30T12:57:45.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed941f5db0 con 0x7fed940ca0d0 2023-10-30T12:57:45.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.618+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed941f68f0 con 0x7fed940ca0d0 2023-10-30T12:57:45.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.622+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed84004b70 con 0x7fed940ca0d0 2023-10-30T12:57:45.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.622+0000 7fed8a7fc640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 0x7fed78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.622+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fed840832f0 con 0x7fed940ca0d0 2023-10-30T12:57:45.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.622+0000 7fed99a2d640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 0x7fed78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.622+0000 7fed99a2d640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 0x7fed78068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed900ae000 tx=0x7fed900a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.626+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed840504c0 con 0x7fed940ca0d0 2023-10-30T12:57:45.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.790+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fed9415a540 con 0x7fed780664a0 2023-10-30T12:57:45.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.790+0000 7fed8a7fc640 1 -- 172.21.15.8:0/3583482394 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fed900aa960 con 0x7fed780664a0 2023-10-30T12:57:45.797 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 5 to deep-scrub 2023-10-30T12:57:45.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 msgr2=0x7fed78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 0x7fed78068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed900ae000 tx=0x7fed900a7000 comp rx=0 tx=0).stop 2023-10-30T12:57:45.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 msgr2=0x7fed94145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c490 tx=0x7fed8400c950 comp rx=0 tx=0).stop 2023-10-30T12:57:45.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 shutdown_connections 2023-10-30T12:57:45.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fed780664a0 0x7fed78068960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed940ca4b0 0x7fed94147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fed940ca0d0 0x7fed94145430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 --2- 172.21.15.8:0/3583482394 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fed940c5560 0x7fed9414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 >> 172.21.15.8:0/3583482394 conn(0x7fed9406bf20 msgr2=0x7fed94146a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 shutdown_connections 2023-10-30T12:57:45.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.794+0000 7fed9aa2f640 1 -- 172.21.15.8:0/3583482394 wait complete. 2023-10-30T12:57:45.824 INFO:tasks.ceph:Scrubbing osd.6 2023-10-30T12:57:45.824 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:45.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 Processor -- start 2023-10-30T12:57:45.976 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 -- start start 2023-10-30T12:57:45.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 0x7efc4806fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.977 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc480c74d0 con 0x7efc480c0930 2023-10-30T12:57:45.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc480718a0 con 0x7efc48150690 2023-10-30T12:57:45.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4e30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc48072420 con 0x7efc480c7e80 2023-10-30T12:57:45.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc477fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7efc480c0930 0x7efc480c7770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38052/0 (socket says 172.21.15.8:38052) 2023-10-30T12:57:45.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc477fe640 1 -- 172.21.15.8:0/2125970793 learned_addr learned my addr 172.21.15.8:0/2125970793 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:45.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/2125970793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 0x7efc4806fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.979 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048843629 0 0) 0x7efc3c001120 con 0x7efc480c0930 2023-10-30T12:57:45.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc30003530 con 0x7efc480c0930 2023-10-30T12:57:45.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163136778 0 0) 0x7efc480718a0 con 0x7efc48150690 2023-10-30T12:57:45.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c001120 con 0x7efc48150690 2023-10-30T12:57:45.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4c8ac640 1 -- 172.21.15.8:0/2125970793 >> v1:172.21.15.46:6789/0 conn(0x7efc48150690 legacy=0x7efc48156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.970+0000 7efc4c8ac640 1 -- 172.21.15.8:0/2125970793 >> v1:172.21.15.8:6789/0 conn(0x7efc480c0930 legacy=0x7efc480c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362896490 0 0) 0x7efc30003530 con 0x7efc480c0930 2023-10-30T12:57:45.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4c8ac640 1 -- 172.21.15.8:0/2125970793 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48071b80 con 0x7efc480c7e80 2023-10-30T12:57:45.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/2125970793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 0x7efc4806fe60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc40009dc0 tx=0x7efc4001f080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96b9075c407418f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:45.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c001810 con 0x7efc480c7e80 2023-10-30T12:57:45.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc400a8c10 con 0x7efc480c7e80 2023-10-30T12:57:45.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc46ffd640 1 -- 172.21.15.8:0/2125970793 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400b0440 con 0x7efc480c7e80 2023-10-30T12:57:45.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 -- 172.21.15.8:0/2125970793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 msgr2=0x7efc4806fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:45.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 --2- 172.21.15.8:0/2125970793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 0x7efc4806fe60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc40009dc0 tx=0x7efc4001f080 comp rx=0 tx=0).stop 2023-10-30T12:57:45.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 -- 172.21.15.8:0/2125970793 shutdown_connections 2023-10-30T12:57:45.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 --2- 172.21.15.8:0/2125970793 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc480c7e80 0x7efc4806fe60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:45.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 -- 172.21.15.8:0/2125970793 >> 172.21.15.8:0/2125970793 conn(0x7efc4806e150 msgr2=0x7efc48069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:45.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.974+0000 7efc4e30f640 1 -- 172.21.15.8:0/2125970793 shutdown_connections 2023-10-30T12:57:45.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 -- 172.21.15.8:0/2125970793 wait complete. 2023-10-30T12:57:45.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 Processor -- start 2023-10-30T12:57:45.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 -- start start 2023-10-30T12:57:45.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc480c0930 0x7efc481e1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 0x7efc481effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc4814c6b0 0x7efc481e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:45.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48155760 con 0x7efc48150690 2023-10-30T12:57:45.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4814b200 con 0x7efc480c0930 2023-10-30T12:57:45.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4e30f640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc48154780 con 0x7efc4814c6b0 2023-10-30T12:57:45.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 0x7efc481effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 0x7efc481effd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48340/0 (socket says 172.21.15.8:48340) 2023-10-30T12:57:45.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 -- 172.21.15.8:0/1621735259 learned_addr learned my addr 172.21.15.8:0/1621735259 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:45.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc47fff640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc480c0930 0x7efc481e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc4814c6b0 0x7efc481e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:45.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 -- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc4814c6b0 msgr2=0x7efc481e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc4814c6b0 0x7efc481e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 -- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc480c0930 msgr2=0x7efc481e1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc480c0930 0x7efc481e1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc477fe640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40009a70 con 0x7efc48150690 2023-10-30T12:57:46.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.978+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7efc4814c6b0 0x7efc481e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:46.002 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc477fe640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 0x7efc481effd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00caf0 tx=0x7efc3c001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c005cf0 con 0x7efc48150690 2023-10-30T12:57:46.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c005e70 con 0x7efc48150690 2023-10-30T12:57:46.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c007290 con 0x7efc48150690 2023-10-30T12:57:46.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4814ac40 con 0x7efc48150690 2023-10-30T12:57:46.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4814c190 con 0x7efc48150690 2023-10-30T12:57:46.005 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc48072140 con 0x7efc48150690 2023-10-30T12:57:46.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc3c0074b0 con 0x7efc48150690 2023-10-30T12:57:46.006 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc340664a0 0x7efc34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7efc3c083e60 con 0x7efc48150690 2023-10-30T12:57:46.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc47fff640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc340664a0 0x7efc34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7efc3406ec50 0x7efc340710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc340717c0 con 0x7efc3406ec50 2023-10-30T12:57:46.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc3c086440 con 0x7efc48150690 2023-10-30T12:57:46.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7efc3406ec50 0x7efc340710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.982+0000 7efc47fff640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc340664a0 0x7efc34068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc38001720 tx=0x7efc38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.986+0000 7efc4c8ac640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7efc3406ec50 0x7efc340710f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.986+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== osd.6 v2:172.21.15.46:6824/815955451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7efc340717c0 con 0x7efc3406ec50 2023-10-30T12:57:46.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.998+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 --> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efc48071e60 con 0x7efc3406ec50 2023-10-30T12:57:46.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:45.998+0000 7efc44ff9640 1 -- 172.21.15.8:0/1621735259 <== osd.6 v2:172.21.15.46:6824/815955451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efc40003d80 con 0x7efc3406ec50 2023-10-30T12:57:46.013 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:46.014 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:46.014 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:46.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7efc3406ec50 msgr2=0x7efc340710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.46:6824/815955451,v1:172.21.15.46:6825/815955451] conn(0x7efc3406ec50 0x7efc340710f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc340664a0 msgr2=0x7efc34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7efc340664a0 0x7efc34068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc38001720 tx=0x7efc38009040 comp rx=0 tx=0).stop 2023-10-30T12:57:46.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 msgr2=0x7efc481effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 --2- 172.21.15.8:0/1621735259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc48150690 0x7efc481effd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00caf0 tx=0x7efc3c001d60 comp rx=0 tx=0).stop 2023-10-30T12:57:46.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4c8ac640 1 -- 172.21.15.8:0/1621735259 reap_dead start 2023-10-30T12:57:46.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 shutdown_connections 2023-10-30T12:57:46.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 >> 172.21.15.8:0/1621735259 conn(0x7efc4806e150 msgr2=0x7efc480c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 shutdown_connections 2023-10-30T12:57:46.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.002+0000 7efc4e30f640 1 -- 172.21.15.8:0/1621735259 wait complete. 2023-10-30T12:57:46.025 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-30T12:57:46.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.162+0000 7f1afb246640 1 Processor -- start 2023-10-30T12:57:46.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.162+0000 7f1afb246640 1 -- start start 2023-10-30T12:57:46.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1af4150830 0x7f1af406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af40cb7d0 0x7f1af40c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1af40c3e40 0x7f1af4070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af40c2310 con 0x7f1af4150830 2023-10-30T12:57:46.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af4156710 con 0x7f1af40c3e40 2023-10-30T12:57:46.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1afb246640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c4360 con 0x7f1af40cb7d0 2023-10-30T12:57:46.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1af4150830 0x7f1af406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1af4150830 0x7f1af406b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48342/0 (socket says 172.21.15.8:48342) 2023-10-30T12:57:46.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 -- 172.21.15.8:0/3920369414 learned_addr learned my addr 172.21.15.8:0/3920369414 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:46.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af97e3640 1 --2- 172.21.15.8:0/3920369414 >> v2:172.21.15.46:3300/0 conn(0x7f1af40c3e40 0x7f1af4070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af3fff640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af40cb7d0 0x7f1af40c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 -- 172.21.15.8:0/3920369414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af40cb7d0 msgr2=0x7f1af40c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af40cb7d0 0x7f1af40c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 -- 172.21.15.8:0/3920369414 >> v2:172.21.15.46:3300/0 conn(0x7f1af40c3e40 msgr2=0x7f1af4070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/3920369414 >> v2:172.21.15.46:3300/0 conn(0x7f1af40c3e40 0x7f1af4070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.166+0000 7f1af8fe2640 1 -- 172.21.15.8:0/3920369414 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af41569b0 con 0x7f1af4150830 2023-10-30T12:57:46.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af4150830 0x7f1af406b500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1af406fc60 tx=0x7f1ae801ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f180ce46b8cd7dbc server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1af37fe640 1 -- 172.21.15.8:0/3920369414 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80a8cd0 con 0x7f1af4150830 2023-10-30T12:57:46.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1af37fe640 1 -- 172.21.15.8:0/3920369414 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80a8e30 con 0x7f1af4150830 2023-10-30T12:57:46.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1af37fe640 1 -- 172.21.15.8:0/3920369414 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80b0660 con 0x7f1af4150830 2023-10-30T12:57:46.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- 172.21.15.8:0/3920369414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af4150830 msgr2=0x7f1af406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af4150830 0x7f1af406b500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1af406fc60 tx=0x7f1ae801ef00 comp rx=0 tx=0).stop 2023-10-30T12:57:46.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- 172.21.15.8:0/3920369414 shutdown_connections 2023-10-30T12:57:46.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 --2- 172.21.15.8:0/3920369414 >> v2:172.21.15.46:3300/0 conn(0x7f1af40c3e40 0x7f1af4070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af40cb7d0 0x7f1af40c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 --2- 172.21.15.8:0/3920369414 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af4150830 0x7f1af406b500 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- 172.21.15.8:0/3920369414 >> 172.21.15.8:0/3920369414 conn(0x7f1af40cac70 msgr2=0x7f1af40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- 172.21.15.8:0/3920369414 shutdown_connections 2023-10-30T12:57:46.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- 172.21.15.8:0/3920369414 wait complete. 2023-10-30T12:57:46.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 Processor -- start 2023-10-30T12:57:46.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.170+0000 7f1afb246640 1 -- start start 2023-10-30T12:57:46.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 0x7f1af414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af4150830 0x7f1af41ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af415a030 con 0x7f1af40cb7d0 2023-10-30T12:57:46.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4159770 con 0x7f1af40c3e40 2023-10-30T12:57:46.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af414e3b0 con 0x7f1af4150830 2023-10-30T12:57:46.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48344/0 (socket says 172.21.15.8:48344) 2023-10-30T12:57:46.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 -- 172.21.15.8:0/1932276481 learned_addr learned my addr 172.21.15.8:0/1932276481 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:46.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 0x7f1af414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af97e3640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af4150830 0x7f1af41ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 -- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af4150830 msgr2=0x7f1af41ea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af4150830 0x7f1af41ea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 -- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 msgr2=0x7f1af414ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 0x7f1af414ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 -- 172.21.15.8:0/1932276481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80094d0 con 0x7f1af40cb7d0 2023-10-30T12:57:46.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 0x7f1af414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:46.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af3fff640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000d5f0 tx=0x7f1ae000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00125f0 con 0x7f1af40cb7d0 2023-10-30T12:57:46.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0012750 con 0x7f1af40cb7d0 2023-10-30T12:57:46.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0012ae0 con 0x7f1af40cb7d0 2023-10-30T12:57:46.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af414cb50 con 0x7f1af40cb7d0 2023-10-30T12:57:46.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af414d100 con 0x7f1af40cb7d0 2023-10-30T12:57:46.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af404e170 con 0x7f1af40cb7d0 2023-10-30T12:57:46.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.174+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1ae0012c40 con 0x7f1af40cb7d0 2023-10-30T12:57:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.178+0000 7f1af17fa640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 0x7f1ad8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.178+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f1ae00845c0 con 0x7f1af40cb7d0 2023-10-30T12:57:46.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.178+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 0x7f1ad8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.178+0000 7f1af8fe2640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 0x7f1ad8068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80b2000 tx=0x7f1ae8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.182+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1ae00517a0 con 0x7f1af40cb7d0 2023-10-30T12:57:46.294 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:46.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:57:46.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.338+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1af4156f70 con 0x7f1ad80664a0 2023-10-30T12:57:46.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.338+0000 7f1af17fa640 1 -- 172.21.15.8:0/1932276481 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1ae80bf090 con 0x7f1ad80664a0 2023-10-30T12:57:46.346 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 6 to deep-scrub 2023-10-30T12:57:46.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 msgr2=0x7f1ad8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 0x7f1ad8068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80b2000 tx=0x7f1ae8008040 comp rx=0 tx=0).stop 2023-10-30T12:57:46.348 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 msgr2=0x7f1af4155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000d5f0 tx=0x7f1ae000dab0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 shutdown_connections 2023-10-30T12:57:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f1ad80664a0 0x7f1ad8068960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f1af4150830 0x7f1af41ea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1af40cb7d0 0x7f1af4155570 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 --2- 172.21.15.8:0/1932276481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af40c3e40 0x7f1af414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 >> 172.21.15.8:0/1932276481 conn(0x7f1af40cac70 msgr2=0x7f1af4070520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 shutdown_connections 2023-10-30T12:57:46.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.342+0000 7f1afb246640 1 -- 172.21.15.8:0/1932276481 wait complete. 2023-10-30T12:57:46.367 INFO:tasks.ceph:Scrubbing osd.7 2023-10-30T12:57:46.368 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-30T12:57:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.554+0000 7f0aacc91640 1 Processor -- start 2023-10-30T12:57:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.554+0000 7f0aacc91640 1 -- start start 2023-10-30T12:57:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.554+0000 7f0aacc91640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0aa8151730 0x7f0aa80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.554+0000 7f0aacc91640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0aa806b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aacc91640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa8072bf0 con 0x7f0aa8151b10 2023-10-30T12:57:46.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aacc91640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa8158560 con 0x7f0aa8151730 2023-10-30T12:57:46.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aacc91640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8157620 con 0x7f0aa8072740 2023-10-30T12:57:46.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0aa8151b10 0x7f0aa80c5fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38058/0 (socket says 172.21.15.8:38058) 2023-10-30T12:57:46.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 -- 172.21.15.8:0/1137382523 learned_addr learned my addr 172.21.15.8:0/1137382523 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:57:46.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0aa806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa659c640 1 --2- 172.21.15.8:0/1137382523 >> v2:172.21.15.46:3300/0 conn(0x7f0aa8151730 0x7f0aa80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811858786 0 0) 0x7f0aa8072bf0 con 0x7f0aa8151b10 2023-10-30T12:57:46.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a90003530 con 0x7f0aa8151b10 2023-10-30T12:57:46.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 40201600 0 0) 0x7f0a90003530 con 0x7f0aa8151b10 2023-10-30T12:57:46.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 msgr2=0x7f0aa806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0aa806b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 >> v2:172.21.15.46:3300/0 conn(0x7f0aa8151730 msgr2=0x7f0aa80c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 --2- 172.21.15.8:0/1137382523 >> v2:172.21.15.46:3300/0 conn(0x7f0aa8151730 0x7f0aa80c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa81578e0 con 0x7f0aa8151b10 2023-10-30T12:57:46.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f0a9c001680 con 0x7f0aa8151b10 2023-10-30T12:57:46.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 >> v1:172.21.15.8:6789/0 conn(0x7f0aa8151b10 legacy=0x7f0aa80c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0a90003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0a900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a90004d10 0x7f0a90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8158560 con 0x7f0a90003f00 2023-10-30T12:57:46.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8157620 con 0x7f0a90004d10 2023-10-30T12:57:46.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000b9c0 con 0x7f0aa8072740 2023-10-30T12:57:46.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0a900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa659c640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0a90003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a90004d10 0x7f0a90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 -- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 msgr2=0x7f0a90003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa8072740 0x7f0a90003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 -- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a90004d10 msgr2=0x7f0a90005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a90004d10 0x7f0a90005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.558+0000 7f0aa5d9b640 1 -- 172.21.15.8:0/1137382523 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90003530 con 0x7f0a90003f00 2023-10-30T12:57:46.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0a900047d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c210 tx=0x7f0a9c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc705968f63e686d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0a5960 con 0x7f0a90003f00 2023-10-30T12:57:46.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0a5ac0 con 0x7f0a90003f00 2023-10-30T12:57:46.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aa559a640 1 -- 172.21.15.8:0/1137382523 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0058e0 con 0x7f0a90003f00 2023-10-30T12:57:46.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- 172.21.15.8:0/1137382523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 msgr2=0x7f0a900047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 --2- 172.21.15.8:0/1137382523 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0a900047d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c210 tx=0x7f0a9c01cc60 comp rx=0 tx=0).stop 2023-10-30T12:57:46.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aa6d9d640 1 -- 172.21.15.8:0/1137382523 reap_dead start 2023-10-30T12:57:46.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- 172.21.15.8:0/1137382523 shutdown_connections 2023-10-30T12:57:46.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- 172.21.15.8:0/1137382523 >> 172.21.15.8:0/1137382523 conn(0x7f0aa806e150 msgr2=0x7f0aa8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- 172.21.15.8:0/1137382523 shutdown_connections 2023-10-30T12:57:46.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- 172.21.15.8:0/1137382523 wait complete. 2023-10-30T12:57:46.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 Processor -- start 2023-10-30T12:57:46.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.562+0000 7f0aacc91640 1 -- start start 2023-10-30T12:57:46.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0aa81406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa814b1a0 0x7f0aa8140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa8142f40 0x7f0aa8145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa81ebbd0 con 0x7f0a90003f00 2023-10-30T12:57:46.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa81ead70 con 0x7f0aa8142f40 2023-10-30T12:57:46.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa81eb050 con 0x7f0aa814b1a0 2023-10-30T12:57:46.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0aa81406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0aa81406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48366/0 (socket says 172.21.15.8:48366) 2023-10-30T12:57:46.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 -- 172.21.15.8:0/2564871943 learned_addr learned my addr 172.21.15.8:0/2564871943 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:46.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa8142f40 0x7f0aa8145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa814b1a0 0x7f0aa8140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 -- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa814b1a0 msgr2=0x7f0aa8140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa814b1a0 0x7f0aa8140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 -- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa8142f40 msgr2=0x7f0aa8145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa8142f40 0x7f0aa8145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa81eb330 con 0x7f0a90003f00 2023-10-30T12:57:46.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa8142f40 0x7f0aa8145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:46.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0aa814b1a0 0x7f0aa8140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:57:46.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aa659c640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0aa81406b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800ba70 tx=0x7f0a9800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98012050 con 0x7f0a90003f00 2023-10-30T12:57:46.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98013040 con 0x7f0a90003f00 2023-10-30T12:57:46.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa81eb610 con 0x7f0a90003f00 2023-10-30T12:57:46.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98014cd0 con 0x7f0a90003f00 2023-10-30T12:57:46.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa8150df0 con 0x7f0a90003f00 2023-10-30T12:57:46.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0aa8157ea0 con 0x7f0a90003f00 2023-10-30T12:57:46.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.566+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a98019440 con 0x7f0a90003f00 2023-10-30T12:57:46.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0a880664a0 0x7f0a88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f0a980887f0 con 0x7f0a90003f00 2023-10-30T12:57:46.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f0a8806ec50 0x7f0a880710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0a880664a0 0x7f0a88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a880717c0 con 0x7f0a8806ec50 2023-10-30T12:57:46.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a9808add0 con 0x7f0a90003f00 2023-10-30T12:57:46.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f0a8806ec50 0x7f0a880710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0aa5d9b640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0a880664a0 0x7f0a88068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0aa000 tx=0x7f0a9c001a10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0aa6d9d640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f0a8806ec50 0x7f0a880710f0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.570+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== osd.7 v2:172.21.15.46:6808/1029552049 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0a880717c0 con 0x7f0a8806ec50 2023-10-30T12:57:46.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 --> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0aa8157bc0 con 0x7f0a8806ec50 2023-10-30T12:57:46.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0a96ffd640 1 -- 172.21.15.8:0/2564871943 <== osd.7 v2:172.21.15.46:6808/1029552049 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0aa000f040 con 0x7f0a8806ec50 2023-10-30T12:57:46.593 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-10-30T12:57:46.594 INFO:teuthology.orchestra.run.smithi008.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T12:57:46.594 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-10-30T12:57:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f0a8806ec50 msgr2=0x7f0a880710f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.46:6808/1029552049,v1:172.21.15.46:6809/1029552049] conn(0x7f0a8806ec50 0x7f0a880710f0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0a880664a0 msgr2=0x7f0a88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f0a880664a0 0x7f0a88068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0aa000 tx=0x7f0a9c001a10 comp rx=0 tx=0).stop 2023-10-30T12:57:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 msgr2=0x7f0aa81406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 --2- 172.21.15.8:0/2564871943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a90003f00 0x7f0aa81406b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800ba70 tx=0x7f0a9800bf30 comp rx=0 tx=0).stop 2023-10-30T12:57:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aa6d9d640 1 -- 172.21.15.8:0/2564871943 reap_dead start 2023-10-30T12:57:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 shutdown_connections 2023-10-30T12:57:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 >> 172.21.15.8:0/2564871943 conn(0x7f0aa806e150 msgr2=0x7f0aa81553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 shutdown_connections 2023-10-30T12:57:46.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.586+0000 7f0aacc91640 1 -- 172.21.15.8:0/2564871943 wait complete. 2023-10-30T12:57:46.610 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-30T12:57:46.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.774+0000 7f6eee3c5640 1 Processor -- start 2023-10-30T12:57:46.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.774+0000 7f6eee3c5640 1 -- start start 2023-10-30T12:57:46.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eee3c5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6ee8153c80 0x7f6ee815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eee3c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee80c4a10 0x7f6ee80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eee3c5640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee8159f20 con 0x7f6ee815a3b0 2023-10-30T12:57:46.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eee3c5640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee815dec0 con 0x7f6ee8153c80 2023-10-30T12:57:46.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eee3c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee81599a0 con 0x7f6ee80c4a10 2023-10-30T12:57:46.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6ee8153c80 0x7f6ee815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6ee8153c80 0x7f6ee815cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48370/0 (socket says 172.21.15.8:48370) 2023-10-30T12:57:46.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 -- 172.21.15.8:0/53059445 learned_addr learned my addr 172.21.15.8:0/53059445 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:46.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6eec962640 1 --1- 172.21.15.8:0/53059445 >> v1:172.21.15.8:6789/0 conn(0x7f6ee815a3b0 0x7f6ee80ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:38064/0 (socket says 172.21.15.8:38064) 2023-10-30T12:57:46.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee77fe640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee80c4a10 0x7f6ee80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 -- 172.21.15.8:0/53059445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee80c4a10 msgr2=0x7f6ee80c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:46.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee80c4a10 0x7f6ee80c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 -- 172.21.15.8:0/53059445 >> v1:172.21.15.8:6789/0 conn(0x7f6ee815a3b0 legacy=0x7f6ee80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.778+0000 7f6ee7fff640 1 -- 172.21.15.8:0/53059445 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80c47e0 con 0x7f6ee8153c80 2023-10-30T12:57:46.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6ee7fff640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee8153c80 0x7f6ee815cf90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009870 tx=0x7f6ed801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a09521c8bb71091 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6ee6ffd640 1 -- 172.21.15.8:0/53059445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80a8a30 con 0x7f6ee8153c80 2023-10-30T12:57:46.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6ee6ffd640 1 -- 172.21.15.8:0/53059445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed80a8b90 con 0x7f6ee8153c80 2023-10-30T12:57:46.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6ee6ffd640 1 -- 172.21.15.8:0/53059445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80b0440 con 0x7f6ee8153c80 2023-10-30T12:57:46.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 -- 172.21.15.8:0/53059445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee8153c80 msgr2=0x7f6ee815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee8153c80 0x7f6ee815cf90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009870 tx=0x7f6ed801ed30 comp rx=0 tx=0).stop 2023-10-30T12:57:46.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 -- 172.21.15.8:0/53059445 shutdown_connections 2023-10-30T12:57:46.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee80c4a10 0x7f6ee80c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/53059445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee8153c80 0x7f6ee815cf90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 -- 172.21.15.8:0/53059445 >> 172.21.15.8:0/53059445 conn(0x7f6ee80715c0 msgr2=0x7f6ee80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 -- 172.21.15.8:0/53059445 shutdown_connections 2023-10-30T12:57:46.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.782+0000 7f6eee3c5640 1 -- 172.21.15.8:0/53059445 wait complete. 2023-10-30T12:57:46.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 Processor -- start 2023-10-30T12:57:46.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- start start 2023-10-30T12:57:46.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ee80c4a10 0x7f6ee8149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee8153c80 0x7f6ee80c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80b2870 con 0x7f6ee815a3b0 2023-10-30T12:57:46.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80b20b0 con 0x7f6ee80c4a10 2023-10-30T12:57:46.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee81527a0 con 0x7f6ee8153c80 2023-10-30T12:57:46.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48376/0 (socket says 172.21.15.8:48376) 2023-10-30T12:57:46.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 -- 172.21.15.8:0/1027917591 learned_addr learned my addr 172.21.15.8:0/1027917591 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:46.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6ee77fe640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee8153c80 0x7f6ee80c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6ee7fff640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ee80c4a10 0x7f6ee8149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 -- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee8153c80 msgr2=0x7f6ee80c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.798 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee8153c80 0x7f6ee80c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 -- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ee80c4a10 msgr2=0x7f6ee8149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ee80c4a10 0x7f6ee8149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 -- 172.21.15.8:0/1027917591 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80094d0 con 0x7f6ee815a3b0 2023-10-30T12:57:46.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eec962640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c700 tx=0x7f6ee000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000ce10 con 0x7f6ee815a3b0 2023-10-30T12:57:46.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000a640 con 0x7f6ee815a3b0 2023-10-30T12:57:46.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000a9b0 con 0x7f6ee815a3b0 2023-10-30T12:57:46.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8151b90 con 0x7f6ee815a3b0 2023-10-30T12:57:46.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.786+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8152150 con 0x7f6ee815a3b0 2023-10-30T12:57:46.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee804e170 con 0x7f6ee815a3b0 2023-10-30T12:57:46.801 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6ee000ab10 con 0x7f6ee815a3b0 2023-10-30T12:57:46.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6ee4ff9640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 0x7f6ed4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:46.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f6ee00837b0 con 0x7f6ee815a3b0 2023-10-30T12:57:46.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6ee7fff640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 0x7f6ed4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:46.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.790+0000 7f6ee7fff640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 0x7f6ed4068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80b2000 tx=0x7f6ed8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:46.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.798+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6ee00544d0 con 0x7f6ee815a3b0 2023-10-30T12:57:46.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.954+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ee815d900 con 0x7f6ed40664a0 2023-10-30T12:57:46.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.954+0000 7f6ee4ff9640 1 -- 172.21.15.8:0/1027917591 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6ed80b0440 con 0x7f6ed40664a0 2023-10-30T12:57:46.962 INFO:teuthology.orchestra.run.smithi008.stderr:instructed osd(s) 7 to deep-scrub 2023-10-30T12:57:46.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 msgr2=0x7f6ed4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 0x7f6ed4068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80b2000 tx=0x7f6ed8008040 comp rx=0 tx=0).stop 2023-10-30T12:57:46.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 msgr2=0x7f6ee80cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:46.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c700 tx=0x7f6ee000cbc0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 shutdown_connections 2023-10-30T12:57:46.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f6ed40664a0 0x7f6ed4068960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee815a3b0 0x7f6ee80cf270 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f6ee8153c80 0x7f6ee80c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 --2- 172.21.15.8:0/1027917591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ee80c4a10 0x7f6ee8149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:46.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 >> 172.21.15.8:0/1027917591 conn(0x7f6ee80715c0 msgr2=0x7f6ee80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:46.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 shutdown_connections 2023-10-30T12:57:46.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:46.958+0000 7f6eee3c5640 1 -- 172.21.15.8:0/1027917591 wait complete. 2023-10-30T12:57:46.983 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:57:47.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 Processor -- start 2023-10-30T12:57:47.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 -- start start 2023-10-30T12:57:47.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc82015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc8200c4a10 0x7fc8200c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc82015a3b0 0x7fc8200ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82015dec0 con 0x7fc8200c4a10 2023-10-30T12:57:47.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8201599a0 con 0x7fc82015a3b0 2023-10-30T12:57:47.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc82493e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc820159f20 con 0x7fc820153c80 2023-10-30T12:57:47.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc8200c4a10 0x7fc8200c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc8200c4a10 0x7fc8200c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48388/0 (socket says 172.21.15.8:48388) 2023-10-30T12:57:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 -- 172.21.15.8:0/861431723 learned_addr learned my addr 172.21.15.8:0/861431723 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81e59c640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc82015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81ed9d640 1 --2- 172.21.15.8:0/861431723 >> v2:172.21.15.46:3300/0 conn(0x7fc82015a3b0 0x7fc8200ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 -- 172.21.15.8:0/861431723 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 msgr2=0x7fc82015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc82015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 -- 172.21.15.8:0/861431723 >> v2:172.21.15.46:3300/0 conn(0x7fc82015a3b0 msgr2=0x7fc8200ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:57:47.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 --2- 172.21.15.8:0/861431723 >> v2:172.21.15.46:3300/0 conn(0x7fc82015a3b0 0x7fc8200ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.126+0000 7fc81dd9b640 1 -- 172.21.15.8:0/861431723 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200c47e0 con 0x7fc8200c4a10 2023-10-30T12:57:47.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc81dd9b640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8200c4a10 0x7fc8200c42a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8200cbbc0 tx=0x7fc81401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c8e0af450d31b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc81d59a640 1 -- 172.21.15.8:0/861431723 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140a89b0 con 0x7fc8200c4a10 2023-10-30T12:57:47.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc81d59a640 1 -- 172.21.15.8:0/861431723 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140a8b10 con 0x7fc8200c4a10 2023-10-30T12:57:47.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc81d59a640 1 -- 172.21.15.8:0/861431723 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140a8ea0 con 0x7fc8200c4a10 2023-10-30T12:57:47.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 -- 172.21.15.8:0/861431723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8200c4a10 msgr2=0x7fc8200c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8200c4a10 0x7fc8200c42a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8200cbbc0 tx=0x7fc81401ecf0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 -- 172.21.15.8:0/861431723 shutdown_connections 2023-10-30T12:57:47.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 --2- 172.21.15.8:0/861431723 >> v2:172.21.15.46:3300/0 conn(0x7fc82015a3b0 0x7fc8200ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8200c4a10 0x7fc8200c42a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 --2- 172.21.15.8:0/861431723 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc82015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 -- 172.21.15.8:0/861431723 >> 172.21.15.8:0/861431723 conn(0x7fc8200715c0 msgr2=0x7fc8200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:47.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 -- 172.21.15.8:0/861431723 shutdown_connections 2023-10-30T12:57:47.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.130+0000 7fc82493e640 1 -- 172.21.15.8:0/861431723 wait complete. 2023-10-30T12:57:47.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 Processor -- start 2023-10-30T12:57:47.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- start start 2023-10-30T12:57:47.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8200c4a10 0x7fc820151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc8200b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200b2b70 con 0x7fc82015a3b0 2023-10-30T12:57:47.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201eea50 con 0x7fc8200c4a10 2023-10-30T12:57:47.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200b2e30 con 0x7fc820153c80 2023-10-30T12:57:47.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48396/0 (socket says 172.21.15.8:48396) 2023-10-30T12:57:47.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 -- 172.21.15.8:0/3926482745 learned_addr learned my addr 172.21.15.8:0/3926482745 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:57:47.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81e59c640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8200c4a10 0x7fc820151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81dd9b640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc8200b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 -- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 msgr2=0x7fc8200b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc8200b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 -- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8200c4a10 msgr2=0x7fc820151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8200c4a10 0x7fc820151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 -- 172.21.15.8:0/3926482745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140094d0 con 0x7fc82015a3b0 2023-10-30T12:57:47.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc81ed9d640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc818007e60 tx=0x7fc81800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818014420 con 0x7fc82015a3b0 2023-10-30T12:57:47.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180145a0 con 0x7fc82015a3b0 2023-10-30T12:57:47.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8201eed10 con 0x7fc82015a3b0 2023-10-30T12:57:47.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818014950 con 0x7fc82015a3b0 2023-10-30T12:57:47.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.134+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8201ef2d0 con 0x7fc82015a3b0 2023-10-30T12:57:47.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8201ef4c0 con 0x7fc82015a3b0 2023-10-30T12:57:47.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc818014ab0 con 0x7fc82015a3b0 2023-10-30T12:57:47.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc80effd640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 0x7fc8000689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:57:47.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fc818084870 con 0x7fc82015a3b0 2023-10-30T12:57:47.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc81e59c640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 0x7fc8000689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:57:47.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.138+0000 7fc81e59c640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 0x7fc8000689e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8100045f0 tx=0x7fc810008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:57:47.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.142+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc818051b50 con 0x7fc82015a3b0 2023-10-30T12:57:47.303 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.298+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc82015d900 con 0x7fc800066520 2023-10-30T12:57:47.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.306+0000 7fc80effd640 1 -- 172.21.15.8:0/3926482745 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290910 (secure 0 0 0) 0x7fc81000b740 con 0x7fc800066520 2023-10-30T12:57:47.314 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:57:47.316 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:57:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 msgr2=0x7fc8000689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 0x7fc8000689e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8100045f0 tx=0x7fc810008040 comp rx=0 tx=0).stop 2023-10-30T12:57:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 msgr2=0x7fc820149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:57:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc818007e60 tx=0x7fc81800e4b0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 shutdown_connections 2023-10-30T12:57:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc800066520 0x7fc8000689e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc82015a3b0 0x7fc820149b10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc820153c80 0x7fc8200b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 --2- 172.21.15.8:0/3926482745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8200c4a10 0x7fc820151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:57:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 >> 172.21.15.8:0/3926482745 conn(0x7fc8200715c0 msgr2=0x7fc820158450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:57:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 shutdown_connections 2023-10-30T12:57:47.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:57:47.310+0000 7fc82493e640 1 -- 172.21.15.8:0/3926482745 wait complete. 2023-10-30T12:57:47.335 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2023-10-30T12:57:45.808490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1102689590,"num_objects":10479,"num_object_clones":0,"num_object_copies":20958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10479,"num_whiteouts":0,"num_read":2548,"num_read_kb":243548,"num_write":26555,"num_write_kb":1228593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92800,"num_omap_keys":171,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27286,"ondisk_log_size":27286,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2704644,"kb_used_data":2209540,"kb_used_omap":12,"kb_used_meta":495027,"kb_avail":752270076,"statfs":{"total":773094113280,"available":770324557824,"internally_reserved":0,"allocated":2262568960,"data_stored":2206259769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":506908218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4520198,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":42,"num_read_kb":46,"num_write":254,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92800,"num_omap_keys":171,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"3.3f","version":"53'341","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678230+0000","last_change":"2023-10-30T12:52:38.167461+0000","last_active":"2023-10-30T12:57:39.678230+0000","last_peered":"2023-10-30T12:57:39.678230+0000","last_clean":"2023-10-30T12:57:39.678230+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:57:39.678230+0000","last_undegraded":"2023-10-30T12:57:39.678230+0000","last_fullsized":"2023-10-30T12:57:39.678230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:46:23.473692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'336","reported_seq":2447,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.602570+0000","last_change":"2023-10-30T12:52:38.158597+0000","last_active":"2023-10-30T12:57:39.602570+0000","last_peered":"2023-10-30T12:57:39.602570+0000","last_clean":"2023-10-30T12:57:39.602570+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:57:39.602570+0000","last_undegraded":"2023-10-30T12:57:39.602570+0000","last_fullsized":"2023-10-30T12:57:39.602570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'359","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621999+0000","last_change":"2023-10-30T12:52:38.161971+0000","last_active":"2023-10-30T12:57:39.621999+0000","last_peered":"2023-10-30T12:57:39.621999+0000","last_clean":"2023-10-30T12:57:39.621999+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:39.621999+0000","last_undegraded":"2023-10-30T12:57:39.621999+0000","last_fullsized":"2023-10-30T12:57:39.621999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'301","reported_seq":2414,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.676973+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:39.676973+0000","last_peered":"2023-10-30T12:57:39.676973+0000","last_clean":"2023-10-30T12:57:39.676973+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:39.676973+0000","last_undegraded":"2023-10-30T12:57:39.676973+0000","last_fullsized":"2023-10-30T12:57:39.676973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:36:39.214543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'351","reported_seq":2464,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671168+0000","last_change":"2023-10-30T12:52:38.166749+0000","last_active":"2023-10-30T12:57:39.671168+0000","last_peered":"2023-10-30T12:57:39.671168+0000","last_clean":"2023-10-30T12:57:39.671168+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:57:39.671168+0000","last_undegraded":"2023-10-30T12:57:39.671168+0000","last_fullsized":"2023-10-30T12:57:39.671168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:39:27.480500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28213093,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":332,"num_write_kb":27731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'332","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672529+0000","last_change":"2023-10-30T12:52:38.190432+0000","last_active":"2023-10-30T12:57:39.672529+0000","last_peered":"2023-10-30T12:57:39.672529+0000","last_clean":"2023-10-30T12:57:39.672529+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:57:39.672529+0000","last_undegraded":"2023-10-30T12:57:39.672529+0000","last_fullsized":"2023-10-30T12:57:39.672529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:33:12.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'373","reported_seq":2479,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672807+0000","last_change":"2023-10-30T12:52:38.164109+0000","last_active":"2023-10-30T12:57:39.672807+0000","last_peered":"2023-10-30T12:57:39.672807+0000","last_clean":"2023-10-30T12:57:39.672807+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:39.672807+0000","last_undegraded":"2023-10-30T12:57:39.672807+0000","last_fullsized":"2023-10-30T12:57:39.672807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:03:58.478040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23819931,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":356,"num_write_kb":23455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'316","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.673026+0000","last_change":"2023-10-30T12:52:38.375236+0000","last_active":"2023-10-30T12:57:39.673026+0000","last_peered":"2023-10-30T12:57:39.673026+0000","last_clean":"2023-10-30T12:57:39.673026+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:57:39.673026+0000","last_undegraded":"2023-10-30T12:57:39.673026+0000","last_fullsized":"2023-10-30T12:57:39.673026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'338","reported_seq":2465,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.679266+0000","last_change":"2023-10-30T12:52:38.168173+0000","last_active":"2023-10-30T12:57:39.679266+0000","last_peered":"2023-10-30T12:57:39.679266+0000","last_clean":"2023-10-30T12:57:39.679266+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:57:39.679266+0000","last_undegraded":"2023-10-30T12:57:39.679266+0000","last_fullsized":"2023-10-30T12:57:39.679266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:17:54.844918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767488,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":327,"num_write_kb":17709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'339","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677329+0000","last_change":"2023-10-30T12:52:38.960608+0000","last_active":"2023-10-30T12:57:39.677329+0000","last_peered":"2023-10-30T12:57:39.677329+0000","last_clean":"2023-10-30T12:57:39.677329+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:57:39.677329+0000","last_undegraded":"2023-10-30T12:57:39.677329+0000","last_fullsized":"2023-10-30T12:57:39.677329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:44:35.999124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'337","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642756+0000","last_change":"2023-10-30T12:52:38.959629+0000","last_active":"2023-10-30T12:57:39.642756+0000","last_peered":"2023-10-30T12:57:39.642756+0000","last_clean":"2023-10-30T12:57:39.642756+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:57:39.642756+0000","last_undegraded":"2023-10-30T12:57:39.642756+0000","last_fullsized":"2023-10-30T12:57:39.642756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:11:54.888888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'312","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674922+0000","last_change":"2023-10-30T12:52:38.378139+0000","last_active":"2023-10-30T12:57:39.674922+0000","last_peered":"2023-10-30T12:57:39.674922+0000","last_clean":"2023-10-30T12:57:39.674922+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:39.674922+0000","last_undegraded":"2023-10-30T12:57:39.674922+0000","last_fullsized":"2023-10-30T12:57:39.674922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:34.995464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'314","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.636559+0000","last_change":"2023-10-30T12:52:38.374275+0000","last_active":"2023-10-30T12:57:39.636559+0000","last_peered":"2023-10-30T12:57:39.636559+0000","last_clean":"2023-10-30T12:57:39.636559+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:57:39.636559+0000","last_undegraded":"2023-10-30T12:57:39.636559+0000","last_fullsized":"2023-10-30T12:57:39.636559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'353","reported_seq":2469,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.959869+0000","last_change":"2023-10-30T12:52:38.161570+0000","last_active":"2023-10-30T12:57:39.959869+0000","last_peered":"2023-10-30T12:57:39.959869+0000","last_clean":"2023-10-30T12:57:39.959869+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:57:39.959869+0000","last_undegraded":"2023-10-30T12:57:39.959869+0000","last_fullsized":"2023-10-30T12:57:39.959869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'344","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660843+0000","last_change":"2023-10-30T12:52:38.157652+0000","last_active":"2023-10-30T12:57:39.660843+0000","last_peered":"2023-10-30T12:57:39.660843+0000","last_clean":"2023-10-30T12:57:39.660843+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:57:39.660843+0000","last_undegraded":"2023-10-30T12:57:39.660843+0000","last_fullsized":"2023-10-30T12:57:39.660843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'372","reported_seq":2495,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.676993+0000","last_change":"2023-10-30T12:52:38.190778+0000","last_active":"2023-10-30T12:57:39.676993+0000","last_peered":"2023-10-30T12:57:39.676993+0000","last_clean":"2023-10-30T12:57:39.676993+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:57:39.676993+0000","last_undegraded":"2023-10-30T12:57:39.676993+0000","last_fullsized":"2023-10-30T12:57:39.676993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:23.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23758433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":359,"num_write_kb":26630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'338","reported_seq":2456,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677890+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:39.677890+0000","last_peered":"2023-10-30T12:57:39.677890+0000","last_clean":"2023-10-30T12:57:39.677890+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:39.677890+0000","last_undegraded":"2023-10-30T12:57:39.677890+0000","last_fullsized":"2023-10-30T12:57:39.677890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:25:39.852314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10013359,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":323,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'346","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638267+0000","last_change":"2023-10-30T12:52:38.708281+0000","last_active":"2023-10-30T12:57:39.638267+0000","last_peered":"2023-10-30T12:57:39.638267+0000","last_clean":"2023-10-30T12:57:39.638267+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:57:39.638267+0000","last_undegraded":"2023-10-30T12:57:39.638267+0000","last_fullsized":"2023-10-30T12:57:39.638267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'361","reported_seq":2473,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674350+0000","last_change":"2023-10-30T12:52:38.166885+0000","last_active":"2023-10-30T12:57:39.674350+0000","last_peered":"2023-10-30T12:57:39.674350+0000","last_clean":"2023-10-30T12:57:39.674350+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:57:39.674350+0000","last_undegraded":"2023-10-30T12:57:39.674350+0000","last_fullsized":"2023-10-30T12:57:39.674350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:17:23.403084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695113,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":341,"num_write_kb":18629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'351","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670046+0000","last_change":"2023-10-30T12:52:38.959805+0000","last_active":"2023-10-30T12:57:39.670046+0000","last_peered":"2023-10-30T12:57:39.670046+0000","last_clean":"2023-10-30T12:57:39.670046+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:57:39.670046+0000","last_undegraded":"2023-10-30T12:57:39.670046+0000","last_fullsized":"2023-10-30T12:57:39.670046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:13:45.750836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'355","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671849+0000","last_change":"2023-10-30T12:52:38.159940+0000","last_active":"2023-10-30T12:57:39.671849+0000","last_peered":"2023-10-30T12:57:39.671849+0000","last_clean":"2023-10-30T12:57:39.671849+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:57:39.671849+0000","last_undegraded":"2023-10-30T12:57:39.671849+0000","last_fullsized":"2023-10-30T12:57:39.671849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'383","reported_seq":2507,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671285+0000","last_change":"2023-10-30T12:52:38.707969+0000","last_active":"2023-10-30T12:57:39.671285+0000","last_peered":"2023-10-30T12:57:39.671285+0000","last_clean":"2023-10-30T12:57:39.671285+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:57:39.671285+0000","last_undegraded":"2023-10-30T12:57:39.671285+0000","last_fullsized":"2023-10-30T12:57:39.671285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'372","reported_seq":2502,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660039+0000","last_change":"2023-10-30T12:52:38.374161+0000","last_active":"2023-10-30T12:57:39.660039+0000","last_peered":"2023-10-30T12:57:39.660039+0000","last_clean":"2023-10-30T12:57:39.660039+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:57:39.660039+0000","last_undegraded":"2023-10-30T12:57:39.660039+0000","last_fullsized":"2023-10-30T12:57:39.660039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'361","reported_seq":2473,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.676926+0000","last_change":"2023-10-30T12:52:38.192255+0000","last_active":"2023-10-30T12:57:39.676926+0000","last_peered":"2023-10-30T12:57:39.676926+0000","last_clean":"2023-10-30T12:57:39.676926+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:57:39.676926+0000","last_undegraded":"2023-10-30T12:57:39.676926+0000","last_fullsized":"2023-10-30T12:57:39.676926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:37.588996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'327","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658441+0000","last_change":"2023-10-30T12:52:38.374594+0000","last_active":"2023-10-30T12:57:39.658441+0000","last_peered":"2023-10-30T12:57:39.658441+0000","last_clean":"2023-10-30T12:57:39.658441+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:57:39.658441+0000","last_undegraded":"2023-10-30T12:57:39.658441+0000","last_fullsized":"2023-10-30T12:57:39.658441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'379","reported_seq":2510,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.669068+0000","last_change":"2023-10-30T12:52:38.960525+0000","last_active":"2023-10-30T12:57:39.669068+0000","last_peered":"2023-10-30T12:57:39.669068+0000","last_clean":"2023-10-30T12:57:39.669068+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:57:39.669068+0000","last_undegraded":"2023-10-30T12:57:39.669068+0000","last_fullsized":"2023-10-30T12:57:39.669068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:09.235440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617342+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:39.617342+0000","last_peered":"2023-10-30T12:57:39.617342+0000","last_clean":"2023-10-30T12:57:39.617342+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:39.617342+0000","last_undegraded":"2023-10-30T12:57:39.617342+0000","last_fullsized":"2023-10-30T12:57:39.617342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'306","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670885+0000","last_change":"2023-10-30T12:52:38.375028+0000","last_active":"2023-10-30T12:57:39.670885+0000","last_peered":"2023-10-30T12:57:39.670885+0000","last_clean":"2023-10-30T12:57:39.670885+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:57:39.670885+0000","last_undegraded":"2023-10-30T12:57:39.670885+0000","last_fullsized":"2023-10-30T12:57:39.670885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'330","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621752+0000","last_change":"2023-10-30T12:52:38.160968+0000","last_active":"2023-10-30T12:57:39.621752+0000","last_peered":"2023-10-30T12:57:39.621752+0000","last_clean":"2023-10-30T12:57:39.621752+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:39.621752+0000","last_undegraded":"2023-10-30T12:57:39.621752+0000","last_fullsized":"2023-10-30T12:57:39.621752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'375","reported_seq":2495,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672601+0000","last_change":"2023-10-30T12:52:38.193081+0000","last_active":"2023-10-30T12:57:39.672601+0000","last_peered":"2023-10-30T12:57:39.672601+0000","last_clean":"2023-10-30T12:57:39.672601+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:57:39.672601+0000","last_undegraded":"2023-10-30T12:57:39.672601+0000","last_fullsized":"2023-10-30T12:57:39.672601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:19.937261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15318011,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":358,"num_write_kb":15146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'376","reported_seq":2498,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.642895+0000","last_change":"2023-10-30T12:52:38.708132+0000","last_active":"2023-10-30T12:57:39.642895+0000","last_peered":"2023-10-30T12:57:39.642895+0000","last_clean":"2023-10-30T12:57:39.642895+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:57:39.642895+0000","last_undegraded":"2023-10-30T12:57:39.642895+0000","last_fullsized":"2023-10-30T12:57:39.642895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'337","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656399+0000","last_change":"2023-10-30T12:52:38.956531+0000","last_active":"2023-10-30T12:57:39.656399+0000","last_peered":"2023-10-30T12:57:39.656399+0000","last_clean":"2023-10-30T12:57:39.656399+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:57:39.656399+0000","last_undegraded":"2023-10-30T12:57:39.656399+0000","last_fullsized":"2023-10-30T12:57:39.656399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:39:37.995081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'342","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.657965+0000","last_change":"2023-10-30T12:52:38.377927+0000","last_active":"2023-10-30T12:57:39.657965+0000","last_peered":"2023-10-30T12:57:39.657965+0000","last_clean":"2023-10-30T12:57:39.657965+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:39.657965+0000","last_undegraded":"2023-10-30T12:57:39.657965+0000","last_fullsized":"2023-10-30T12:57:39.657965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:50:29.073770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"36'262","reported_seq":319,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.273023+0000","last_change":"2023-10-30T12:45:18.114401+0000","last_active":"2023-10-30T12:57:12.273023+0000","last_peered":"2023-10-30T12:57:12.273023+0000","last_clean":"2023-10-30T12:57:12.273023+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:12.273023+0000","last_undegraded":"2023-10-30T12:57:12.273023+0000","last_fullsized":"2023-10-30T12:57:12.273023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'324","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658123+0000","last_change":"2023-10-30T12:52:38.161345+0000","last_active":"2023-10-30T12:57:39.658123+0000","last_peered":"2023-10-30T12:57:39.658123+0000","last_clean":"2023-10-30T12:57:39.658123+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:39.658123+0000","last_undegraded":"2023-10-30T12:57:39.658123+0000","last_fullsized":"2023-10-30T12:57:39.658123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'293","reported_seq":417,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.556766+0000","last_change":"2023-10-30T12:45:18.102593+0000","last_active":"2023-10-30T12:57:41.556766+0000","last_peered":"2023-10-30T12:57:41.556766+0000","last_clean":"2023-10-30T12:57:41.556766+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:57:41.556766+0000","last_undegraded":"2023-10-30T12:57:41.556766+0000","last_fullsized":"2023-10-30T12:57:41.556766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:52:37.090250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10824096,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":290,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"52'72","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:24.783983+0000","last_change":"2023-10-30T12:45:18.098274+0000","last_active":"2023-10-30T12:57:24.783983+0000","last_peered":"2023-10-30T12:57:24.783983+0000","last_clean":"2023-10-30T12:57:24.783983+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:24.783983+0000","last_undegraded":"2023-10-30T12:57:24.783983+0000","last_fullsized":"2023-10-30T12:57:24.783983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'400","reported_seq":2532,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.613163+0000","last_change":"2023-10-30T12:52:38.375331+0000","last_active":"2023-10-30T12:57:39.613163+0000","last_peered":"2023-10-30T12:57:39.613163+0000","last_clean":"2023-10-30T12:57:39.613163+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:39.613163+0000","last_undegraded":"2023-10-30T12:57:39.613163+0000","last_fullsized":"2023-10-30T12:57:39.613163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"52'341","reported_seq":392,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.262243+0000","last_change":"2023-10-30T12:45:18.111809+0000","last_active":"2023-10-30T12:57:12.262243+0000","last_peered":"2023-10-30T12:57:12.262243+0000","last_clean":"2023-10-30T12:57:12.262243+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:12.262243+0000","last_undegraded":"2023-10-30T12:57:12.262243+0000","last_fullsized":"2023-10-30T12:57:12.262243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:41:31.535396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'356","reported_seq":2478,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617156+0000","last_change":"2023-10-30T12:52:38.157348+0000","last_active":"2023-10-30T12:57:39.617156+0000","last_peered":"2023-10-30T12:57:39.617156+0000","last_clean":"2023-10-30T12:57:39.617156+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:57:39.617156+0000","last_undegraded":"2023-10-30T12:57:39.617156+0000","last_fullsized":"2023-10-30T12:57:39.617156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'31","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:29.786928+0000","last_change":"2023-10-30T12:45:18.109093+0000","last_active":"2023-10-30T12:57:29.786928+0000","last_peered":"2023-10-30T12:57:29.786928+0000","last_clean":"2023-10-30T12:57:29.786928+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:29.786928+0000","last_undegraded":"2023-10-30T12:57:29.786928+0000","last_fullsized":"2023-10-30T12:57:29.786928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:34:36.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":31,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'367","reported_seq":2491,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658306+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:39.658306+0000","last_peered":"2023-10-30T12:57:39.658306+0000","last_clean":"2023-10-30T12:57:39.658306+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:39.658306+0000","last_undegraded":"2023-10-30T12:57:39.658306+0000","last_fullsized":"2023-10-30T12:57:39.658306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:32:09.875012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'190","reported_seq":244,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.446857+0000","last_change":"2023-10-30T12:45:18.103252+0000","last_active":"2023-10-30T12:57:41.446857+0000","last_peered":"2023-10-30T12:57:41.446857+0000","last_clean":"2023-10-30T12:57:41.446857+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:57:41.446857+0000","last_undegraded":"2023-10-30T12:57:41.446857+0000","last_fullsized":"2023-10-30T12:57:41.446857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:16.725023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1991030,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":1051,"num_write":189,"num_write_kb":2076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'349","reported_seq":2478,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.622611+0000","last_change":"2023-10-30T12:52:38.162408+0000","last_active":"2023-10-30T12:57:39.622611+0000","last_peered":"2023-10-30T12:57:39.622611+0000","last_clean":"2023-10-30T12:57:39.622611+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:57:39.622611+0000","last_undegraded":"2023-10-30T12:57:39.622611+0000","last_fullsized":"2023-10-30T12:57:39.622611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"52'43","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:24.792983+0000","last_change":"2023-10-30T12:45:18.089131+0000","last_active":"2023-10-30T12:57:24.792983+0000","last_peered":"2023-10-30T12:57:24.792983+0000","last_clean":"2023-10-30T12:57:24.792983+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:24.792983+0000","last_undegraded":"2023-10-30T12:57:24.792983+0000","last_fullsized":"2023-10-30T12:57:24.792983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'332","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674677+0000","last_change":"2023-10-30T12:52:38.378836+0000","last_active":"2023-10-30T12:57:39.674677+0000","last_peered":"2023-10-30T12:57:39.674677+0000","last_clean":"2023-10-30T12:57:39.674677+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:39.674677+0000","last_undegraded":"2023-10-30T12:57:39.674677+0000","last_fullsized":"2023-10-30T12:57:39.674677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:57:03.390521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'107","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:31.407376+0000","last_change":"2023-10-30T12:45:18.084677+0000","last_active":"2023-10-30T12:57:31.407376+0000","last_peered":"2023-10-30T12:57:31.407376+0000","last_clean":"2023-10-30T12:57:31.407376+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:31.407376+0000","last_undegraded":"2023-10-30T12:57:31.407376+0000","last_fullsized":"2023-10-30T12:57:31.407376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:21:11.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":106,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'317","reported_seq":2422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671610+0000","last_change":"2023-10-30T12:52:38.163983+0000","last_active":"2023-10-30T12:57:39.671610+0000","last_peered":"2023-10-30T12:57:39.671610+0000","last_clean":"2023-10-30T12:57:39.671610+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:57:39.671610+0000","last_undegraded":"2023-10-30T12:57:39.671610+0000","last_fullsized":"2023-10-30T12:57:39.671610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:25:26.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"53'363","reported_seq":434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.626957+0000","last_change":"2023-10-30T12:57:44.627005+0000","last_active":"2023-10-30T12:57:44.626957+0000","last_peered":"2023-10-30T12:57:44.626957+0000","last_clean":"2023-10-30T12:57:44.626957+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:44.626957+0000","last_undegraded":"2023-10-30T12:57:44.626957+0000","last_fullsized":"2023-10-30T12:57:44.626957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'363","last_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_deep_scrub":"53'363","last_deep_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.626874+0000","objects_scrubbed":9,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.857541+0000","scrub_duration":0.036431423999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31162,"num_omap_keys":58,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'359","reported_seq":2486,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671638+0000","last_change":"2023-10-30T12:52:38.161686+0000","last_active":"2023-10-30T12:57:39.671638+0000","last_peered":"2023-10-30T12:57:39.671638+0000","last_clean":"2023-10-30T12:57:39.671638+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:39.671638+0000","last_undegraded":"2023-10-30T12:57:39.671638+0000","last_fullsized":"2023-10-30T12:57:39.671638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"52'104","reported_seq":161,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.315208+0000","last_change":"2023-10-30T12:45:18.102285+0000","last_active":"2023-10-30T12:57:35.315208+0000","last_peered":"2023-10-30T12:57:35.315208+0000","last_clean":"2023-10-30T12:57:35.315208+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:35.315208+0000","last_undegraded":"2023-10-30T12:57:35.315208+0000","last_fullsized":"2023-10-30T12:57:35.315208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:15:45.364306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":7391,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'357","reported_seq":2494,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668393+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:39.668393+0000","last_peered":"2023-10-30T12:57:39.668393+0000","last_clean":"2023-10-30T12:57:39.668393+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:39.668393+0000","last_undegraded":"2023-10-30T12:57:39.668393+0000","last_fullsized":"2023-10-30T12:57:39.668393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:18:51.506289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'399","reported_seq":458,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.487499+0000","last_change":"2023-10-30T12:45:18.112339+0000","last_active":"2023-10-30T12:57:41.487499+0000","last_peered":"2023-10-30T12:57:41.487499+0000","last_clean":"2023-10-30T12:57:41.487499+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:57:41.487499+0000","last_undegraded":"2023-10-30T12:57:41.487499+0000","last_fullsized":"2023-10-30T12:57:41.487499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:45:46.260091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":392,"num_write_kb":16593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'331","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.656466+0000","last_change":"2023-10-30T12:52:38.959038+0000","last_active":"2023-10-30T12:57:39.656466+0000","last_peered":"2023-10-30T12:57:39.656466+0000","last_clean":"2023-10-30T12:57:39.656466+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:57:39.656466+0000","last_undegraded":"2023-10-30T12:57:39.656466+0000","last_fullsized":"2023-10-30T12:57:39.656466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:03:23.132047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"53'125","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.181850+0000","last_change":"2023-10-30T12:57:44.181906+0000","last_active":"2023-10-30T12:57:44.181850+0000","last_peered":"2023-10-30T12:57:44.181850+0000","last_clean":"2023-10-30T12:57:44.181850+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:44.181850+0000","last_undegraded":"2023-10-30T12:57:44.181850+0000","last_fullsized":"2023-10-30T12:57:44.181850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.181771+0000","objects_scrubbed":13,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:04:46.051796+0000","scrub_duration":0.017190094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51810,"num_omap_keys":95,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'357","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.753366+0000","last_change":"2023-10-30T12:52:38.707521+0000","last_active":"2023-10-30T12:57:39.753366+0000","last_peered":"2023-10-30T12:57:39.753366+0000","last_clean":"2023-10-30T12:57:39.753366+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:57:39.753366+0000","last_undegraded":"2023-10-30T12:57:39.753366+0000","last_fullsized":"2023-10-30T12:57:39.753366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'103","reported_seq":158,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.900877+0000","last_change":"2023-10-30T12:45:18.094569+0000","last_active":"2023-10-30T12:57:19.900877+0000","last_peered":"2023-10-30T12:57:19.900877+0000","last_clean":"2023-10-30T12:57:19.900877+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:19.900877+0000","last_undegraded":"2023-10-30T12:57:19.900877+0000","last_fullsized":"2023-10-30T12:57:19.900877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:02.544510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":178,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'312","reported_seq":2432,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668473+0000","last_change":"2023-10-30T12:52:38.166069+0000","last_active":"2023-10-30T12:57:39.668473+0000","last_peered":"2023-10-30T12:57:39.668473+0000","last_clean":"2023-10-30T12:57:39.668473+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:57:39.668473+0000","last_undegraded":"2023-10-30T12:57:39.668473+0000","last_fullsized":"2023-10-30T12:57:39.668473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:36.932929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12072627,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":307,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"52'165","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.609377+0000","last_change":"2023-10-30T12:45:18.088683+0000","last_active":"2023-10-30T12:57:07.609377+0000","last_peered":"2023-10-30T12:57:07.609377+0000","last_clean":"2023-10-30T12:57:07.609377+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:57:07.609377+0000","last_undegraded":"2023-10-30T12:57:07.609377+0000","last_fullsized":"2023-10-30T12:57:07.609377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'339","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678306+0000","last_change":"2023-10-30T12:52:38.167701+0000","last_active":"2023-10-30T12:57:39.678306+0000","last_peered":"2023-10-30T12:57:39.678306+0000","last_clean":"2023-10-30T12:57:39.678306+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:57:39.678306+0000","last_undegraded":"2023-10-30T12:57:39.678306+0000","last_fullsized":"2023-10-30T12:57:39.678306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:02:54.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"52'123","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.239970+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:57:07.239970+0000","last_peered":"2023-10-30T12:57:07.239970+0000","last_clean":"2023-10-30T12:57:07.239970+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:57:07.239970+0000","last_undegraded":"2023-10-30T12:57:07.239970+0000","last_fullsized":"2023-10-30T12:57:07.239970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'367","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672319+0000","last_change":"2023-10-30T12:52:38.194662+0000","last_active":"2023-10-30T12:57:39.672319+0000","last_peered":"2023-10-30T12:57:39.672319+0000","last_clean":"2023-10-30T12:57:39.672319+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:57:39.672319+0000","last_undegraded":"2023-10-30T12:57:39.672319+0000","last_fullsized":"2023-10-30T12:57:39.672319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:33:24.813894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526751,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":358,"num_write_kb":14373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"52'12","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.609849+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:57:07.609849+0000","last_peered":"2023-10-30T12:57:07.609849+0000","last_clean":"2023-10-30T12:57:07.609849+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:57:07.609849+0000","last_undegraded":"2023-10-30T12:57:07.609849+0000","last_fullsized":"2023-10-30T12:57:07.609849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'333","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678183+0000","last_change":"2023-10-30T12:52:38.166263+0000","last_active":"2023-10-30T12:57:39.678183+0000","last_peered":"2023-10-30T12:57:39.678183+0000","last_clean":"2023-10-30T12:57:39.678183+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:57:39.678183+0000","last_undegraded":"2023-10-30T12:57:39.678183+0000","last_fullsized":"2023-10-30T12:57:39.678183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:42:44.813227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468449,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":323,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.243330+0000","last_change":"2023-10-30T12:45:07.076382+0000","last_active":"2023-10-30T12:57:07.243330+0000","last_peered":"2023-10-30T12:57:07.243330+0000","last_clean":"2023-10-30T12:57:07.243330+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:07.243330+0000","last_undegraded":"2023-10-30T12:57:07.243330+0000","last_fullsized":"2023-10-30T12:57:07.243330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:05.672131+0000","last_clean_scrub_stamp":"2023-10-30T12:45:05.672131+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:36:49.677998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'10","reported_seq":67,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:13.616235+0000","last_change":"2023-10-30T12:45:18.103747+0000","last_active":"2023-10-30T12:57:13.616235+0000","last_peered":"2023-10-30T12:57:13.616235+0000","last_clean":"2023-10-30T12:57:13.616235+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:13.616235+0000","last_undegraded":"2023-10-30T12:57:13.616235+0000","last_fullsized":"2023-10-30T12:57:13.616235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:14:46.429849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'310","reported_seq":2427,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.614203+0000","last_change":"2023-10-30T12:52:38.374620+0000","last_active":"2023-10-30T12:57:39.614203+0000","last_peered":"2023-10-30T12:57:39.614203+0000","last_clean":"2023-10-30T12:57:39.614203+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:57:39.614203+0000","last_undegraded":"2023-10-30T12:57:39.614203+0000","last_fullsized":"2023-10-30T12:57:39.614203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"52'497","reported_seq":557,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:12.260199+0000","last_change":"2023-10-30T12:45:18.117910+0000","last_active":"2023-10-30T12:57:12.260199+0000","last_peered":"2023-10-30T12:57:12.260199+0000","last_clean":"2023-10-30T12:57:12.260199+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:12.260199+0000","last_undegraded":"2023-10-30T12:57:12.260199+0000","last_fullsized":"2023-10-30T12:57:12.260199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:20.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'327","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.663181+0000","last_change":"2023-10-30T12:52:38.159894+0000","last_active":"2023-10-30T12:57:39.663181+0000","last_peered":"2023-10-30T12:57:39.663181+0000","last_clean":"2023-10-30T12:57:39.663181+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:57:39.663181+0000","last_undegraded":"2023-10-30T12:57:39.663181+0000","last_fullsized":"2023-10-30T12:57:39.663181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'140","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.533827+0000","last_change":"2023-10-30T12:45:18.086583+0000","last_active":"2023-10-30T12:57:41.533827+0000","last_peered":"2023-10-30T12:57:41.533827+0000","last_clean":"2023-10-30T12:57:41.533827+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:57:41.533827+0000","last_undegraded":"2023-10-30T12:57:41.533827+0000","last_fullsized":"2023-10-30T12:57:41.533827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4531037,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":136,"num_write_kb":4573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'325","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617538+0000","last_change":"2023-10-30T12:52:38.157754+0000","last_active":"2023-10-30T12:57:39.617538+0000","last_peered":"2023-10-30T12:57:39.617538+0000","last_clean":"2023-10-30T12:57:39.617538+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:57:39.617538+0000","last_undegraded":"2023-10-30T12:57:39.617538+0000","last_fullsized":"2023-10-30T12:57:39.617538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"52'245","reported_seq":297,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:35.453118+0000","last_change":"2023-10-30T12:45:18.104705+0000","last_active":"2023-10-30T12:56:35.453118+0000","last_peered":"2023-10-30T12:56:35.453118+0000","last_clean":"2023-10-30T12:56:35.453118+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:56:35.453118+0000","last_undegraded":"2023-10-30T12:56:35.453118+0000","last_fullsized":"2023-10-30T12:56:35.453118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:32:21.036231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'373","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.654586+0000","last_change":"2023-10-30T12:52:38.380527+0000","last_active":"2023-10-30T12:57:39.654586+0000","last_peered":"2023-10-30T12:57:39.654586+0000","last_clean":"2023-10-30T12:57:39.654586+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:39.654586+0000","last_undegraded":"2023-10-30T12:57:39.654586+0000","last_fullsized":"2023-10-30T12:57:39.654586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:28:52.581209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'3","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:13.615000+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:57:13.615000+0000","last_peered":"2023-10-30T12:57:13.615000+0000","last_clean":"2023-10-30T12:57:13.615000+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:57:13.615000+0000","last_undegraded":"2023-10-30T12:57:13.615000+0000","last_fullsized":"2023-10-30T12:57:13.615000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'330","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672842+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:39.672842+0000","last_peered":"2023-10-30T12:57:39.672842+0000","last_clean":"2023-10-30T12:57:39.672842+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:39.672842+0000","last_undegraded":"2023-10-30T12:57:39.672842+0000","last_fullsized":"2023-10-30T12:57:39.672842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:40:35.262822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9698880,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":313,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'10","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.415044+0000","last_change":"2023-10-30T12:45:18.121017+0000","last_active":"2023-10-30T12:57:19.415044+0000","last_peered":"2023-10-30T12:57:19.415044+0000","last_clean":"2023-10-30T12:57:19.415044+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:19.415044+0000","last_undegraded":"2023-10-30T12:57:19.415044+0000","last_fullsized":"2023-10-30T12:57:19.415044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:15.172432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'341","reported_seq":2462,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621151+0000","last_change":"2023-10-30T12:52:38.158164+0000","last_active":"2023-10-30T12:57:39.621151+0000","last_peered":"2023-10-30T12:57:39.621151+0000","last_clean":"2023-10-30T12:57:39.621151+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:57:39.621151+0000","last_undegraded":"2023-10-30T12:57:39.621151+0000","last_fullsized":"2023-10-30T12:57:39.621151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'100","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.406486+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:57:41.406486+0000","last_peered":"2023-10-30T12:57:41.406486+0000","last_clean":"2023-10-30T12:57:41.406486+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:57:41.406486+0000","last_undegraded":"2023-10-30T12:57:41.406486+0000","last_fullsized":"2023-10-30T12:57:41.406486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":240,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'314","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672296+0000","last_change":"2023-10-30T12:52:38.374909+0000","last_active":"2023-10-30T12:57:39.672296+0000","last_peered":"2023-10-30T12:57:39.672296+0000","last_clean":"2023-10-30T12:57:39.672296+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:57:39.672296+0000","last_undegraded":"2023-10-30T12:57:39.672296+0000","last_fullsized":"2023-10-30T12:57:39.672296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"52'12","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.610156+0000","last_change":"2023-10-30T12:45:18.117935+0000","last_active":"2023-10-30T12:57:07.610156+0000","last_peered":"2023-10-30T12:57:07.610156+0000","last_clean":"2023-10-30T12:57:07.610156+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:07.610156+0000","last_undegraded":"2023-10-30T12:57:07.610156+0000","last_fullsized":"2023-10-30T12:57:07.610156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:51:30.280910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'331","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.621641+0000","last_change":"2023-10-30T12:52:38.161992+0000","last_active":"2023-10-30T12:57:39.621641+0000","last_peered":"2023-10-30T12:57:39.621641+0000","last_clean":"2023-10-30T12:57:39.621641+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:39.621641+0000","last_undegraded":"2023-10-30T12:57:39.621641+0000","last_fullsized":"2023-10-30T12:57:39.621641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'163","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.332459+0000","last_change":"2023-10-30T12:45:18.117578+0000","last_active":"2023-10-30T12:57:35.332459+0000","last_peered":"2023-10-30T12:57:35.332459+0000","last_clean":"2023-10-30T12:57:35.332459+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:35.332459+0000","last_undegraded":"2023-10-30T12:57:35.332459+0000","last_fullsized":"2023-10-30T12:57:35.332459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:53:51.513211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'340","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668698+0000","last_change":"2023-10-30T12:52:38.380106+0000","last_active":"2023-10-30T12:57:39.668698+0000","last_peered":"2023-10-30T12:57:39.668698+0000","last_clean":"2023-10-30T12:57:39.668698+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:39.668698+0000","last_undegraded":"2023-10-30T12:57:39.668698+0000","last_fullsized":"2023-10-30T12:57:39.668698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:12:33.391336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"52'58","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.418607+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:57:35.418607+0000","last_peered":"2023-10-30T12:57:35.418607+0000","last_clean":"2023-10-30T12:57:35.418607+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:57:35.418607+0000","last_undegraded":"2023-10-30T12:57:35.418607+0000","last_fullsized":"2023-10-30T12:57:35.418607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'327","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.603249+0000","last_change":"2023-10-30T12:52:38.707846+0000","last_active":"2023-10-30T12:57:39.603249+0000","last_peered":"2023-10-30T12:57:39.603249+0000","last_clean":"2023-10-30T12:57:39.603249+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:57:39.603249+0000","last_undegraded":"2023-10-30T12:57:39.603249+0000","last_fullsized":"2023-10-30T12:57:39.603249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'119","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:40.286952+0000","last_change":"2023-10-30T12:45:18.083982+0000","last_active":"2023-10-30T12:57:40.286952+0000","last_peered":"2023-10-30T12:57:40.286952+0000","last_clean":"2023-10-30T12:57:40.286952+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:57:40.286952+0000","last_undegraded":"2023-10-30T12:57:40.286952+0000","last_fullsized":"2023-10-30T12:57:40.286952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":199,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'330","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.607157+0000","last_change":"2023-10-30T12:52:38.373796+0000","last_active":"2023-10-30T12:57:39.607157+0000","last_peered":"2023-10-30T12:57:39.607157+0000","last_clean":"2023-10-30T12:57:39.607157+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:57:39.607157+0000","last_undegraded":"2023-10-30T12:57:39.607157+0000","last_fullsized":"2023-10-30T12:57:39.607157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"52'187","reported_seq":242,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.245389+0000","last_change":"2023-10-30T12:45:18.121669+0000","last_active":"2023-10-30T12:57:07.245389+0000","last_peered":"2023-10-30T12:57:07.245389+0000","last_clean":"2023-10-30T12:57:07.245389+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:57:07.245389+0000","last_undegraded":"2023-10-30T12:57:07.245389+0000","last_fullsized":"2023-10-30T12:57:07.245389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:18:28.079128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":4107,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'332","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.659101+0000","last_change":"2023-10-30T12:52:38.955966+0000","last_active":"2023-10-30T12:57:39.659101+0000","last_peered":"2023-10-30T12:57:39.659101+0000","last_clean":"2023-10-30T12:57:39.659101+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:57:39.659101+0000","last_undegraded":"2023-10-30T12:57:39.659101+0000","last_fullsized":"2023-10-30T12:57:39.659101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:55.980109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'334","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672660+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:39.672660+0000","last_peered":"2023-10-30T12:57:39.672660+0000","last_clean":"2023-10-30T12:57:39.672660+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:39.672660+0000","last_undegraded":"2023-10-30T12:57:39.672660+0000","last_fullsized":"2023-10-30T12:57:39.672660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:59:52.978943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22516204,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":313,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"52'151","reported_seq":202,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:29.983897+0000","last_change":"2023-10-30T12:45:18.117528+0000","last_active":"2023-10-30T12:56:29.983897+0000","last_peered":"2023-10-30T12:56:29.983897+0000","last_clean":"2023-10-30T12:56:29.983897+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:56:29.983897+0000","last_undegraded":"2023-10-30T12:56:29.983897+0000","last_fullsized":"2023-10-30T12:56:29.983897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:03:39.105417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'335","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.676905+0000","last_change":"2023-10-30T12:52:38.194210+0000","last_active":"2023-10-30T12:57:39.676905+0000","last_peered":"2023-10-30T12:57:39.676905+0000","last_clean":"2023-10-30T12:57:39.676905+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:57:39.676905+0000","last_undegraded":"2023-10-30T12:57:39.676905+0000","last_fullsized":"2023-10-30T12:57:39.676905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:56:10.108457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242846,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":314,"num_write_kb":14255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"52'362","reported_seq":420,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-30T12:56:40.158457+0000","last_change":"2023-10-30T12:45:18.117566+0000","last_active":"2023-10-30T12:56:40.158457+0000","last_peered":"2023-10-30T12:56:40.158457+0000","last_clean":"2023-10-30T12:56:40.158457+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:56:40.158457+0000","last_undegraded":"2023-10-30T12:56:40.158457+0000","last_fullsized":"2023-10-30T12:56:40.158457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:16:40.553906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'335","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.643067+0000","last_change":"2023-10-30T12:52:38.162371+0000","last_active":"2023-10-30T12:57:39.643067+0000","last_peered":"2023-10-30T12:57:39.643067+0000","last_clean":"2023-10-30T12:57:39.643067+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:57:39.643067+0000","last_undegraded":"2023-10-30T12:57:39.643067+0000","last_fullsized":"2023-10-30T12:57:39.643067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"52'278","reported_seq":334,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:19.783426+0000","last_change":"2023-10-30T12:45:18.090232+0000","last_active":"2023-10-30T12:57:19.783426+0000","last_peered":"2023-10-30T12:57:19.783426+0000","last_clean":"2023-10-30T12:57:19.783426+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:19.783426+0000","last_undegraded":"2023-10-30T12:57:19.783426+0000","last_fullsized":"2023-10-30T12:57:19.783426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'331","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638389+0000","last_change":"2023-10-30T12:52:38.373954+0000","last_active":"2023-10-30T12:57:39.638389+0000","last_peered":"2023-10-30T12:57:39.638389+0000","last_clean":"2023-10-30T12:57:39.638389+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:57:39.638389+0000","last_undegraded":"2023-10-30T12:57:39.638389+0000","last_fullsized":"2023-10-30T12:57:39.638389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"53'201","reported_seq":274,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.639230+0000","last_change":"2023-10-30T12:57:43.639282+0000","last_active":"2023-10-30T12:57:43.639230+0000","last_peered":"2023-10-30T12:57:43.639230+0000","last_clean":"2023-10-30T12:57:43.639230+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:57:43.639230+0000","last_undegraded":"2023-10-30T12:57:43.639230+0000","last_fullsized":"2023-10-30T12:57:43.639230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'201","last_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_deep_scrub":"53'201","last_deep_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_clean_scrub_stamp":"2023-10-30T12:57:43.639154+0000","objects_scrubbed":8,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:04.340919+0000","scrub_duration":0.033784741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925184388,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20887,"num_write_kb":1047691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902518272,"data_stored":1848828680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21900,"ondisk_log_size":21900,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":176914834,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":601,"num_read_kb":143265,"num_write":5509,"num_write_kb":178486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92800,"num_omap_keys":171,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":353984512,"data_stored":353829668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5269,"ondisk_log_size":5269,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407316,"kb_used_data":327564,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93964524,"statfs":{"total":96636764160,"available":96219672576,"internally_reserved":0,"allocated":335425536,"data_stored":327567493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":81656267},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":0.97999999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":0.755},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.111}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.042}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.081},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.3069999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":1.3959999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":282672,"kb_used_data":237608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94089168,"statfs":{"total":96636764160,"available":96347308032,"internally_reserved":0,"allocated":243310592,"data_stored":238090429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.0329999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.94899999999999995}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.2470000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1519999999999999},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.2010000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364836,"kb_used_data":293404,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94007004,"statfs":{"total":96636764160,"available":96263172096,"internally_reserved":0,"allocated":300445696,"data_stored":292673924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":0.92400000000000004}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":0.71699999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.246}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417424,"kb_used_data":341832,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93954416,"statfs":{"total":96636764160,"available":96209321984,"internally_reserved":0,"allocated":350035968,"data_stored":341877317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":0.747}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.157}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":1.03},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.119}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.208},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":1.0740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406356,"kb_used_data":334924,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93965484,"statfs":{"total":96636764160,"available":96220655616,"internally_reserved":0,"allocated":342962176,"data_stored":334657996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73136584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.433},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Oct 30 12:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.499},"max":{"1min":0.98099999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83799999999999997},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.226,"5min":1.786,"15min":1.867},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":0.88300000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.90100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":177872,"kb_used_data":150856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94193968,"statfs":{"total":96636764160,"available":96454623232,"internally_reserved":0,"allocated":154476544,"data_stored":149400264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.54400000000000004}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.73999999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.754},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.81100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266344,"kb_used_data":212960,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94105496,"statfs":{"total":96636764160,"available":96364027904,"internally_reserved":0,"allocated":218071040,"data_stored":212483944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.435}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":0.80600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381824,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990016,"statfs":{"total":96636764160,"available":96245776384,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.79500000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.034}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.085}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.1100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12943360,"data_stored":12920373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42483712,"data_stored":42462582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61169664,"data_stored":61155870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35569664,"data_stored":35546840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50765824,"data_stored":50746727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264265728,"data_stored":256824371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":262901760,"data_stored":255370516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:57:47.344 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.345 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.345 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.346 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.346 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.347 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.347 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.347 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.348 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.348 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.349 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.349 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.350 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.350 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.350 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.351 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.351 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.352 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.352 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.352 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.353 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.353 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.353 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.354 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.354 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.355 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.355 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.355 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.356 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.356 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.357 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.357 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.357 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.358 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.358 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.359 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.359 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.359 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.360 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.360 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.361 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.361 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.361 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.362 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.362 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.362 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.363 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.363 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.364 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.364 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.364 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.365 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.365 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.366 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.366 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.366 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.367 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.367 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.367 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.368 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.368 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.369 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.369 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-30T12:45:05.672131+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=5, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.369 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.370 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.370 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.371 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.371 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.371 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.372 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.372 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.373 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.373 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.373 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.374 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.374 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.375 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.375 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.375 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.376 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.376 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.376 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.377 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.377 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.378 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.378 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.378 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.379 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.379 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.380 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.380 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.380 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.381 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.381 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:57:47.382 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:58:07.383 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:58:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.542+0000 7fc8a6c32640 1 Processor -- start 2023-10-30T12:58:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.542+0000 7fc8a6c32640 1 -- start start 2023-10-30T12:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a6c32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a0151840 0x7fc8a00c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a6c32640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc8a00cb8b0 0x7fc8a006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a6c32640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a00c5030 con 0x7fc8a0151c20 2023-10-30T12:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a6c32640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a0157830 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a6c32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0157b10 con 0x7fc8a0151840 2023-10-30T12:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a51cf640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc8a00cb8b0 0x7fc8a006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc89ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc8a0151c20 0x7fc8a0072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:48214/0 (socket says 172.21.15.8:48214) 2023-10-30T12:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc89ffff640 1 -- 172.21.15.8:0/2112103330 learned_addr learned my addr 172.21.15.8:0/2112103330 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a49ce640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a0151840 0x7fc8a00c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a51cf640 1 -- 172.21.15.8:0/2112103330 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a0151840 msgr2=0x7fc8a00c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a51cf640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a0151840 0x7fc8a00c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a51cf640 1 -- 172.21.15.8:0/2112103330 >> v1:172.21.15.46:6789/0 conn(0x7fc8a0151c20 legacy=0x7fc8a0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.546+0000 7fc8a51cf640 1 -- 172.21.15.8:0/2112103330 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0157df0 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a51cf640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a00cb8b0 0x7fc8a006b600 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc898009d90 tx=0x7fc89801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf618294d83aba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:07.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc89f7fe640 1 -- 172.21.15.8:0/2112103330 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980af600 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc89f7fe640 1 -- 172.21.15.8:0/2112103330 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8980ab050 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc89f7fe640 1 -- 172.21.15.8:0/2112103330 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980af980 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 -- 172.21.15.8:0/2112103330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a00cb8b0 msgr2=0x7fc8a006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a00cb8b0 0x7fc8a006b600 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc898009d90 tx=0x7fc89801f130 comp rx=0 tx=0).stop 2023-10-30T12:58:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 -- 172.21.15.8:0/2112103330 shutdown_connections 2023-10-30T12:58:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a00cb8b0 0x7fc8a006b600 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/2112103330 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a0151840 0x7fc8a00c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 -- 172.21.15.8:0/2112103330 >> 172.21.15.8:0/2112103330 conn(0x7fc8a00cac70 msgr2=0x7fc8a00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:07.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 -- 172.21.15.8:0/2112103330 shutdown_connections 2023-10-30T12:58:07.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.550+0000 7fc8a6c32640 1 -- 172.21.15.8:0/2112103330 wait complete. 2023-10-30T12:58:07.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 Processor -- start 2023-10-30T12:58:07.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- start start 2023-10-30T12:58:07.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a00cb8b0 0x7fc8a00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 0x7fc8a01e4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01f2d10 con 0x7fc8a0151840 2023-10-30T12:58:07.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00ca580 con 0x7fc8a0151c20 2023-10-30T12:58:07.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00ca7f0 con 0x7fc8a00cb8b0 2023-10-30T12:58:07.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41366/0 (socket says 172.21.15.8:41366) 2023-10-30T12:58:07.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 -- 172.21.15.8:0/3272276605 learned_addr learned my addr 172.21.15.8:0/3272276605 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:07.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a49ce640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a00cb8b0 0x7fc8a00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a51cf640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 0x7fc8a01e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 -- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a00cb8b0 msgr2=0x7fc8a00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a00cb8b0 0x7fc8a00ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 -- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 msgr2=0x7fc8a01e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 0x7fc8a01e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 -- 172.21.15.8:0/3272276605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898009a70 con 0x7fc8a0151840 2023-10-30T12:58:07.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a51cf640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 0x7fc8a01e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:07.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89ffff640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc88c001ec0 tx=0x7fc88c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:07.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c005cf0 con 0x7fc8a0151840 2023-10-30T12:58:07.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c005e50 con 0x7fc8a0151840 2023-10-30T12:58:07.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0072d0 con 0x7fc8a0151840 2023-10-30T12:58:07.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a01f2100 con 0x7fc8a0151840 2023-10-30T12:58:07.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a01f26c0 con 0x7fc8a0151840 2023-10-30T12:58:07.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.554+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a00726b0 con 0x7fc8a0151840 2023-10-30T12:58:07.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.558+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc88c007430 con 0x7fc8a0151840 2023-10-30T12:58:07.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.558+0000 7fc89d7fa640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 0x7fc884068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:07.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.558+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fc88c0836d0 con 0x7fc8a0151840 2023-10-30T12:58:07.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.558+0000 7fc8a49ce640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 0x7fc884068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:07.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.558+0000 7fc8a49ce640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 0x7fc884068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc894009b60 tx=0x7fc894009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:07.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.562+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc88c050830 con 0x7fc8a0151840 2023-10-30T12:58:07.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.730+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8a01583b0 con 0x7fc8840664a0 2023-10-30T12:58:07.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.738+0000 7fc89d7fa640 1 -- 172.21.15.8:0/3272276605 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291728 (secure 0 0 0) 0x7fc894011ac0 con 0x7fc8840664a0 2023-10-30T12:58:07.747 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:07.749 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:58:07.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 msgr2=0x7fc884068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 0x7fc884068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc894009b60 tx=0x7fc894009190 comp rx=0 tx=0).stop 2023-10-30T12:58:07.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 msgr2=0x7fc8a0145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:07.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc88c001ec0 tx=0x7fc88c00c890 comp rx=0 tx=0).stop 2023-10-30T12:58:07.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 shutdown_connections 2023-10-30T12:58:07.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fc8840664a0 0x7fc884068960 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0151c20 0x7fc8a01e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8a0151840 0x7fc8a0145780 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 --2- 172.21.15.8:0/3272276605 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fc8a00cb8b0 0x7fc8a00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:07.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 >> 172.21.15.8:0/3272276605 conn(0x7fc8a00cac70 msgr2=0x7fc8a00cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:07.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 shutdown_connections 2023-10-30T12:58:07.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:07.742+0000 7fc8a6c32640 1 -- 172.21.15.8:0/3272276605 wait complete. 2023-10-30T12:58:07.767 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2023-10-30T12:58:05.813780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1103294433,"num_objects":10479,"num_object_clones":0,"num_object_copies":20958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10479,"num_whiteouts":0,"num_read":2548,"num_read_kb":243548,"num_write":26721,"num_write_kb":1229236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":556264,"num_omap_keys":1039,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30855,"ondisk_log_size":30855,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2676700,"kb_used_data":2210716,"kb_used_omap":12,"kb_used_meta":465907,"kb_avail":752298020,"statfs":{"total":773094113280,"available":770353172480,"internally_reserved":0,"allocated":2263773184,"data_stored":2207469455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":477089338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":129098,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":337832,"num_omap_keys":635,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"3.3f","version":"53'341","reported_seq":2462,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:46.962835+0000","last_change":"2023-10-30T12:52:38.167461+0000","last_active":"2023-10-30T12:57:46.962835+0000","last_peered":"2023-10-30T12:57:46.962835+0000","last_clean":"2023-10-30T12:57:46.962835+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:57:46.962835+0000","last_undegraded":"2023-10-30T12:57:46.962835+0000","last_fullsized":"2023-10-30T12:57:46.962835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'336","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.619144+0000","last_change":"2023-10-30T12:52:38.158597+0000","last_active":"2023-10-30T12:57:43.619144+0000","last_peered":"2023-10-30T12:57:43.619144+0000","last_clean":"2023-10-30T12:57:43.619144+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:57:43.619144+0000","last_undegraded":"2023-10-30T12:57:43.619144+0000","last_fullsized":"2023-10-30T12:57:43.619144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'510","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.591196+0000","last_change":"2023-10-30T12:57:58.591267+0000","last_active":"2023-10-30T12:57:58.591196+0000","last_peered":"2023-10-30T12:57:58.591196+0000","last_clean":"2023-10-30T12:57:58.591196+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:58.591196+0000","last_undegraded":"2023-10-30T12:57:58.591196+0000","last_fullsized":"2023-10-30T12:57:58.591196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'510","last_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_deep_scrub":"53'510","last_deep_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.591110+0000","objects_scrubbed":152,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:03:49.294530+0000","scrub_duration":0.16537424100000001,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'301","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677349+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:39.677349+0000","last_peered":"2023-10-30T12:57:39.677349+0000","last_clean":"2023-10-30T12:57:39.677349+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:39.677349+0000","last_undegraded":"2023-10-30T12:57:39.677349+0000","last_fullsized":"2023-10-30T12:57:39.677349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'351","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:46.962834+0000","last_change":"2023-10-30T12:52:38.166749+0000","last_active":"2023-10-30T12:57:46.962834+0000","last_peered":"2023-10-30T12:57:46.962834+0000","last_clean":"2023-10-30T12:57:46.962834+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:57:46.962834+0000","last_undegraded":"2023-10-30T12:57:46.962834+0000","last_fullsized":"2023-10-30T12:57:46.962834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28213093,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":332,"num_write_kb":27731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'332","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797961+0000","last_change":"2023-10-30T12:52:38.190432+0000","last_active":"2023-10-30T12:57:45.797961+0000","last_peered":"2023-10-30T12:57:45.797961+0000","last_clean":"2023-10-30T12:57:45.797961+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:57:45.797961+0000","last_undegraded":"2023-10-30T12:57:45.797961+0000","last_fullsized":"2023-10-30T12:57:45.797961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'546","reported_seq":2669,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.122880+0000","last_change":"2023-10-30T12:57:48.122880+0000","last_active":"2023-10-30T12:57:48.122880+0000","last_peered":"2023-10-30T12:57:48.122880+0000","last_clean":"2023-10-30T12:57:48.122880+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:48.122880+0000","last_undegraded":"2023-10-30T12:57:48.122880+0000","last_fullsized":"2023-10-30T12:57:48.122880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'546","last_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_deep_scrub":"53'546","last_deep_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.122543+0000","objects_scrubbed":180,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:00.797869+0000","scrub_duration":0.33340951699999999,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23819931,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":356,"num_write_kb":23455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'449","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.579958+0000","last_change":"2023-10-30T12:58:03.580002+0000","last_active":"2023-10-30T12:58:03.579958+0000","last_peered":"2023-10-30T12:58:03.579958+0000","last_clean":"2023-10-30T12:58:03.579958+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:58:03.579958+0000","last_undegraded":"2023-10-30T12:58:03.579958+0000","last_fullsized":"2023-10-30T12:58:03.579958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.579888+0000","objects_scrubbed":134,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:08:01.622733+0000","scrub_duration":0.15696447399999999,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'338","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.830402+0000","last_change":"2023-10-30T12:52:38.168173+0000","last_active":"2023-10-30T12:57:39.830402+0000","last_peered":"2023-10-30T12:57:39.830402+0000","last_clean":"2023-10-30T12:57:39.830402+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:57:39.830402+0000","last_undegraded":"2023-10-30T12:57:39.830402+0000","last_fullsized":"2023-10-30T12:57:39.830402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767488,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":327,"num_write_kb":17709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'339","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677424+0000","last_change":"2023-10-30T12:52:38.960608+0000","last_active":"2023-10-30T12:57:39.677424+0000","last_peered":"2023-10-30T12:57:39.677424+0000","last_clean":"2023-10-30T12:57:39.677424+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:57:39.677424+0000","last_undegraded":"2023-10-30T12:57:39.677424+0000","last_fullsized":"2023-10-30T12:57:39.677424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'500","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.600687+0000","last_change":"2023-10-30T12:58:03.600771+0000","last_active":"2023-10-30T12:58:03.600687+0000","last_peered":"2023-10-30T12:58:03.600687+0000","last_clean":"2023-10-30T12:58:03.600687+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:58:03.600687+0000","last_undegraded":"2023-10-30T12:58:03.600687+0000","last_fullsized":"2023-10-30T12:58:03.600687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.600594+0000","objects_scrubbed":164,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:38.231319+0000","scrub_duration":0.225903303,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'459","reported_seq":2586,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.648328+0000","last_change":"2023-10-30T12:57:55.648414+0000","last_active":"2023-10-30T12:57:55.648328+0000","last_peered":"2023-10-30T12:57:55.648328+0000","last_clean":"2023-10-30T12:57:55.648328+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:55.648328+0000","last_undegraded":"2023-10-30T12:57:55.648328+0000","last_fullsized":"2023-10-30T12:57:55.648328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'459","last_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_deep_scrub":"53'459","last_deep_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.648231+0000","objects_scrubbed":147,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:45:01.167844+0000","scrub_duration":0.24832579699999999,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'314","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.636559+0000","last_change":"2023-10-30T12:52:38.374275+0000","last_active":"2023-10-30T12:57:39.636559+0000","last_peered":"2023-10-30T12:57:39.636559+0000","last_clean":"2023-10-30T12:57:39.636559+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:57:39.636559+0000","last_undegraded":"2023-10-30T12:57:39.636559+0000","last_fullsized":"2023-10-30T12:57:39.636559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'500","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.561433+0000","last_change":"2023-10-30T12:58:03.561531+0000","last_active":"2023-10-30T12:58:03.561433+0000","last_peered":"2023-10-30T12:58:03.561433+0000","last_clean":"2023-10-30T12:58:03.561433+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:58:03.561433+0000","last_undegraded":"2023-10-30T12:58:03.561433+0000","last_fullsized":"2023-10-30T12:58:03.561433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.561319+0000","objects_scrubbed":148,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:27.469036+0000","scrub_duration":0.14169902500000001,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'344","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.660843+0000","last_change":"2023-10-30T12:52:38.157652+0000","last_active":"2023-10-30T12:57:39.660843+0000","last_peered":"2023-10-30T12:57:39.660843+0000","last_clean":"2023-10-30T12:57:39.660843+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:57:39.660843+0000","last_undegraded":"2023-10-30T12:57:39.660843+0000","last_fullsized":"2023-10-30T12:57:39.660843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'372","reported_seq":2496,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677123+0000","last_change":"2023-10-30T12:52:38.190778+0000","last_active":"2023-10-30T12:57:39.677123+0000","last_peered":"2023-10-30T12:57:39.677123+0000","last_clean":"2023-10-30T12:57:39.677123+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:57:39.677123+0000","last_undegraded":"2023-10-30T12:57:39.677123+0000","last_fullsized":"2023-10-30T12:57:39.677123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23758433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":359,"num_write_kb":26630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'338","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678058+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:39.678058+0000","last_peered":"2023-10-30T12:57:39.678058+0000","last_clean":"2023-10-30T12:57:39.678058+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:39.678058+0000","last_undegraded":"2023-10-30T12:57:39.678058+0000","last_fullsized":"2023-10-30T12:57:39.678058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10013359,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":323,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'346","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638267+0000","last_change":"2023-10-30T12:52:38.708281+0000","last_active":"2023-10-30T12:57:39.638267+0000","last_peered":"2023-10-30T12:57:39.638267+0000","last_clean":"2023-10-30T12:57:39.638267+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:57:39.638267+0000","last_undegraded":"2023-10-30T12:57:39.638267+0000","last_fullsized":"2023-10-30T12:57:39.638267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'361","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674544+0000","last_change":"2023-10-30T12:52:38.166885+0000","last_active":"2023-10-30T12:57:39.674544+0000","last_peered":"2023-10-30T12:57:39.674544+0000","last_clean":"2023-10-30T12:57:39.674544+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:57:39.674544+0000","last_undegraded":"2023-10-30T12:57:39.674544+0000","last_fullsized":"2023-10-30T12:57:39.674544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695113,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":341,"num_write_kb":18629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'351","reported_seq":2469,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670407+0000","last_change":"2023-10-30T12:52:38.959805+0000","last_active":"2023-10-30T12:57:39.670407+0000","last_peered":"2023-10-30T12:57:39.670407+0000","last_clean":"2023-10-30T12:57:39.670407+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:57:39.670407+0000","last_undegraded":"2023-10-30T12:57:39.670407+0000","last_fullsized":"2023-10-30T12:57:39.670407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'355","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671849+0000","last_change":"2023-10-30T12:52:38.159940+0000","last_active":"2023-10-30T12:57:39.671849+0000","last_peered":"2023-10-30T12:57:39.671849+0000","last_clean":"2023-10-30T12:57:39.671849+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:57:39.671849+0000","last_undegraded":"2023-10-30T12:57:39.671849+0000","last_fullsized":"2023-10-30T12:57:39.671849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'383","reported_seq":2507,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.671285+0000","last_change":"2023-10-30T12:52:38.707969+0000","last_active":"2023-10-30T12:57:39.671285+0000","last_peered":"2023-10-30T12:57:39.671285+0000","last_clean":"2023-10-30T12:57:39.671285+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:57:39.671285+0000","last_undegraded":"2023-10-30T12:57:39.671285+0000","last_fullsized":"2023-10-30T12:57:39.671285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'533","reported_seq":2677,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.612587+0000","last_change":"2023-10-30T12:58:01.612683+0000","last_active":"2023-10-30T12:58:01.612587+0000","last_peered":"2023-10-30T12:58:01.612587+0000","last_clean":"2023-10-30T12:58:01.612587+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:58:01.612587+0000","last_undegraded":"2023-10-30T12:58:01.612587+0000","last_fullsized":"2023-10-30T12:58:01.612587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'533","last_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_deep_scrub":"53'533","last_deep_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.612496+0000","objects_scrubbed":161,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:32.894742+0000","scrub_duration":0.16518128200000001,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'534","reported_seq":2661,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.217181+0000","last_change":"2023-10-30T12:58:03.217312+0000","last_active":"2023-10-30T12:58:03.217181+0000","last_peered":"2023-10-30T12:58:03.217181+0000","last_clean":"2023-10-30T12:58:03.217181+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:58:03.217181+0000","last_undegraded":"2023-10-30T12:58:03.217181+0000","last_fullsized":"2023-10-30T12:58:03.217181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'534","last_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_deep_scrub":"53'534","last_deep_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.216977+0000","objects_scrubbed":173,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:50:07.436714+0000","scrub_duration":0.30228984199999998,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'327","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.658441+0000","last_change":"2023-10-30T12:52:38.374594+0000","last_active":"2023-10-30T12:57:39.658441+0000","last_peered":"2023-10-30T12:57:39.658441+0000","last_clean":"2023-10-30T12:57:39.658441+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:57:39.658441+0000","last_undegraded":"2023-10-30T12:57:39.658441+0000","last_fullsized":"2023-10-30T12:57:39.658441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'379","reported_seq":2511,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.669184+0000","last_change":"2023-10-30T12:52:38.960525+0000","last_active":"2023-10-30T12:57:39.669184+0000","last_peered":"2023-10-30T12:57:39.669184+0000","last_clean":"2023-10-30T12:57:39.669184+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:57:39.669184+0000","last_undegraded":"2023-10-30T12:57:39.669184+0000","last_fullsized":"2023-10-30T12:57:39.669184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617342+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:39.617342+0000","last_peered":"2023-10-30T12:57:39.617342+0000","last_clean":"2023-10-30T12:57:39.617342+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:39.617342+0000","last_undegraded":"2023-10-30T12:57:39.617342+0000","last_fullsized":"2023-10-30T12:57:39.617342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'306","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.670885+0000","last_change":"2023-10-30T12:52:38.375028+0000","last_active":"2023-10-30T12:57:39.670885+0000","last_peered":"2023-10-30T12:57:39.670885+0000","last_clean":"2023-10-30T12:57:39.670885+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:57:39.670885+0000","last_undegraded":"2023-10-30T12:57:39.670885+0000","last_fullsized":"2023-10-30T12:57:39.670885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'481","reported_seq":2612,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.751923+0000","last_change":"2023-10-30T12:57:49.751923+0000","last_active":"2023-10-30T12:57:49.751923+0000","last_peered":"2023-10-30T12:57:49.751923+0000","last_clean":"2023-10-30T12:57:49.751923+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:49.751923+0000","last_undegraded":"2023-10-30T12:57:49.751923+0000","last_fullsized":"2023-10-30T12:57:49.751923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'481","last_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_deep_scrub":"53'481","last_deep_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.751798+0000","objects_scrubbed":156,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:28.001220+0000","scrub_duration":0.14818989499999999,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'375","reported_seq":2497,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797803+0000","last_change":"2023-10-30T12:52:38.193081+0000","last_active":"2023-10-30T12:57:45.797803+0000","last_peered":"2023-10-30T12:57:45.797803+0000","last_clean":"2023-10-30T12:57:45.797803+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:57:45.797803+0000","last_undegraded":"2023-10-30T12:57:45.797803+0000","last_fullsized":"2023-10-30T12:57:45.797803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15318011,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":358,"num_write_kb":15146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'376","reported_seq":2499,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.156290+0000","last_change":"2023-10-30T12:52:38.708132+0000","last_active":"2023-10-30T12:57:44.156290+0000","last_peered":"2023-10-30T12:57:44.156290+0000","last_clean":"2023-10-30T12:57:44.156290+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:57:44.156290+0000","last_undegraded":"2023-10-30T12:57:44.156290+0000","last_fullsized":"2023-10-30T12:57:44.156290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'476","reported_seq":2608,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:04.621477+0000","last_change":"2023-10-30T12:58:04.621599+0000","last_active":"2023-10-30T12:58:04.621477+0000","last_peered":"2023-10-30T12:58:04.621477+0000","last_clean":"2023-10-30T12:58:04.621477+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:58:04.621477+0000","last_undegraded":"2023-10-30T12:58:04.621477+0000","last_fullsized":"2023-10-30T12:58:04.621477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_clean_scrub_stamp":"2023-10-30T12:58:04.621266+0000","objects_scrubbed":140,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:22:51.575535+0000","scrub_duration":0.236240279,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'492","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.717128+0000","last_change":"2023-10-30T12:57:50.717216+0000","last_active":"2023-10-30T12:57:50.717128+0000","last_peered":"2023-10-30T12:57:50.717128+0000","last_clean":"2023-10-30T12:57:50.717128+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:50.717128+0000","last_undegraded":"2023-10-30T12:57:50.717128+0000","last_fullsized":"2023-10-30T12:57:50.717128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'492","last_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_deep_scrub":"53'492","last_deep_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.717022+0000","objects_scrubbed":158,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:37.999028+0000","scrub_duration":0.26506830399999998,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"53'267","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.465321+0000","last_change":"2023-10-30T12:57:48.465321+0000","last_active":"2023-10-30T12:57:48.465321+0000","last_peered":"2023-10-30T12:57:48.465321+0000","last_clean":"2023-10-30T12:57:48.465321+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:48.465321+0000","last_undegraded":"2023-10-30T12:57:48.465321+0000","last_fullsized":"2023-10-30T12:57:48.465321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'267","last_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_deep_scrub":"53'267","last_deep_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.465192+0000","objects_scrubbed":5,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:16.586033+0000","scrub_duration":0.036637109000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8500,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'466","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:46.810452+0000","last_change":"2023-10-30T12:57:46.810452+0000","last_active":"2023-10-30T12:57:46.810452+0000","last_peered":"2023-10-30T12:57:46.810452+0000","last_clean":"2023-10-30T12:57:46.810452+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:46.810452+0000","last_undegraded":"2023-10-30T12:57:46.810452+0000","last_fullsized":"2023-10-30T12:57:46.810452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_clean_scrub_stamp":"2023-10-30T12:57:46.810303+0000","objects_scrubbed":147,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:25:30.111932+0000","scrub_duration":0.192780493,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'311","reported_seq":445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:57.477869+0000","last_change":"2023-10-30T12:57:57.478006+0000","last_active":"2023-10-30T12:57:57.477869+0000","last_peered":"2023-10-30T12:57:57.477869+0000","last_clean":"2023-10-30T12:57:57.477869+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:57:57.477869+0000","last_undegraded":"2023-10-30T12:57:57.477869+0000","last_fullsized":"2023-10-30T12:57:57.477869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'311","last_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_deep_scrub":"53'311","last_deep_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.477645+0000","objects_scrubbed":9,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:49:22.878603+0000","scrub_duration":0.042990041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11031975,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":300,"num_write_kb":11001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":49566,"num_omap_keys":93,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"53'74","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.463815+0000","last_change":"2023-10-30T12:57:49.463904+0000","last_active":"2023-10-30T12:57:49.463815+0000","last_peered":"2023-10-30T12:57:49.463815+0000","last_clean":"2023-10-30T12:57:49.463815+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:49.463815+0000","last_undegraded":"2023-10-30T12:57:49.463815+0000","last_fullsized":"2023-10-30T12:57:49.463815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'74","last_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_deep_scrub":"53'74","last_deep_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.463707+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:46:36.321809+0000","scrub_duration":0.0076797799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'574","reported_seq":2721,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.610591+0000","last_change":"2023-10-30T12:57:58.610634+0000","last_active":"2023-10-30T12:57:58.610591+0000","last_peered":"2023-10-30T12:57:58.610591+0000","last_clean":"2023-10-30T12:57:58.610591+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:58.610591+0000","last_undegraded":"2023-10-30T12:57:58.610591+0000","last_fullsized":"2023-10-30T12:57:58.610591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'574","last_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_deep_scrub":"53'574","last_deep_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.610507+0000","objects_scrubbed":176,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:34.361815+0000","scrub_duration":0.18830925700000001,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"53'347","reported_seq":407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.875242+0000","last_change":"2023-10-30T12:57:45.875338+0000","last_active":"2023-10-30T12:57:45.875242+0000","last_peered":"2023-10-30T12:57:45.875242+0000","last_clean":"2023-10-30T12:57:45.875242+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:45.875242+0000","last_undegraded":"2023-10-30T12:57:45.875242+0000","last_fullsized":"2023-10-30T12:57:45.875242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'347","last_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_deep_scrub":"53'347","last_deep_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.875104+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:05.779308+0000","scrub_duration":0.036461027999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37674,"num_omap_keys":69,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'356","reported_seq":2478,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617156+0000","last_change":"2023-10-30T12:52:38.157348+0000","last_active":"2023-10-30T12:57:39.617156+0000","last_peered":"2023-10-30T12:57:39.617156+0000","last_clean":"2023-10-30T12:57:39.617156+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:57:39.617156+0000","last_undegraded":"2023-10-30T12:57:39.617156+0000","last_fullsized":"2023-10-30T12:57:39.617156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'34","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.423530+0000","last_change":"2023-10-30T12:57:58.423647+0000","last_active":"2023-10-30T12:57:58.423530+0000","last_peered":"2023-10-30T12:57:58.423530+0000","last_clean":"2023-10-30T12:57:58.423530+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:58.423530+0000","last_undegraded":"2023-10-30T12:57:58.423530+0000","last_fullsized":"2023-10-30T12:57:58.423530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.423320+0000","objects_scrubbed":3,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:35.335688+0000","scrub_duration":0.0090485700000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":32,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9500,"num_omap_keys":19,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'367","reported_seq":2492,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672397+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:39.672397+0000","last_peered":"2023-10-30T12:57:39.672397+0000","last_clean":"2023-10-30T12:57:39.672397+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:39.672397+0000","last_undegraded":"2023-10-30T12:57:39.672397+0000","last_fullsized":"2023-10-30T12:57:39.672397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'335","reported_seq":398,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:57.930091+0000","last_change":"2023-10-30T12:57:57.930204+0000","last_active":"2023-10-30T12:57:57.930091+0000","last_peered":"2023-10-30T12:57:57.930091+0000","last_clean":"2023-10-30T12:57:57.930091+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:57:57.930091+0000","last_undegraded":"2023-10-30T12:57:57.930091+0000","last_fullsized":"2023-10-30T12:57:57.930091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'335","last_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_deep_scrub":"53'335","last_deep_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.929845+0000","objects_scrubbed":4,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:04.674566+0000","scrub_duration":0.019613695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2371127,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":1051,"num_write":331,"num_write_kb":2489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26754,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'519","reported_seq":2662,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.566278+0000","last_change":"2023-10-30T12:58:00.566341+0000","last_active":"2023-10-30T12:58:00.566278+0000","last_peered":"2023-10-30T12:58:00.566278+0000","last_clean":"2023-10-30T12:58:00.566278+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:58:00.566278+0000","last_undegraded":"2023-10-30T12:58:00.566278+0000","last_fullsized":"2023-10-30T12:58:00.566278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'519","last_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_deep_scrub":"53'519","last_deep_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.566191+0000","objects_scrubbed":170,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:02:51.046360+0000","scrub_duration":0.153203224,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"53'49","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.101254+0000","last_change":"2023-10-30T12:57:52.101317+0000","last_active":"2023-10-30T12:57:52.101254+0000","last_peered":"2023-10-30T12:57:52.101254+0000","last_clean":"2023-10-30T12:57:52.101254+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:52.101254+0000","last_undegraded":"2023-10-30T12:57:52.101254+0000","last_fullsized":"2023-10-30T12:57:52.101254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'49","last_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_deep_scrub":"53'49","last_deep_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.101149+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:45.571818+0000","scrub_duration":0.011136024,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26812,"num_omap_keys":51,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'498","reported_seq":2627,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.764619+0000","last_change":"2023-10-30T12:57:53.764679+0000","last_active":"2023-10-30T12:57:53.764619+0000","last_peered":"2023-10-30T12:57:53.764619+0000","last_clean":"2023-10-30T12:57:53.764619+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:53.764619+0000","last_undegraded":"2023-10-30T12:57:53.764619+0000","last_fullsized":"2023-10-30T12:57:53.764619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.764533+0000","objects_scrubbed":168,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:27.949277+0000","scrub_duration":0.31117073299999998,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'113","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.479478+0000","last_change":"2023-10-30T12:57:48.479616+0000","last_active":"2023-10-30T12:57:48.479478+0000","last_peered":"2023-10-30T12:57:48.479478+0000","last_clean":"2023-10-30T12:57:48.479478+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:48.479478+0000","last_undegraded":"2023-10-30T12:57:48.479478+0000","last_fullsized":"2023-10-30T12:57:48.479478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.479247+0000","objects_scrubbed":6,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:36.989062+0000","scrub_duration":0.019741666000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":107,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8868,"num_omap_keys":17,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'317","reported_seq":2423,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.674433+0000","last_change":"2023-10-30T12:52:38.163983+0000","last_active":"2023-10-30T12:57:39.674433+0000","last_peered":"2023-10-30T12:57:39.674433+0000","last_clean":"2023-10-30T12:57:39.674433+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:57:39.674433+0000","last_undegraded":"2023-10-30T12:57:39.674433+0000","last_fullsized":"2023-10-30T12:57:39.674433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"53'363","reported_seq":434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.626957+0000","last_change":"2023-10-30T12:57:44.627005+0000","last_active":"2023-10-30T12:57:44.626957+0000","last_peered":"2023-10-30T12:57:44.626957+0000","last_clean":"2023-10-30T12:57:44.626957+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:44.626957+0000","last_undegraded":"2023-10-30T12:57:44.626957+0000","last_fullsized":"2023-10-30T12:57:44.626957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'363","last_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_deep_scrub":"53'363","last_deep_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.626874+0000","objects_scrubbed":9,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.857541+0000","scrub_duration":0.036431423999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31162,"num_omap_keys":58,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'521","reported_seq":2663,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.826274+0000","last_change":"2023-10-30T12:57:45.826319+0000","last_active":"2023-10-30T12:57:45.826274+0000","last_peered":"2023-10-30T12:57:45.826274+0000","last_clean":"2023-10-30T12:57:45.826274+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:45.826274+0000","last_undegraded":"2023-10-30T12:57:45.826274+0000","last_fullsized":"2023-10-30T12:57:45.826274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'521","last_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_deep_scrub":"53'521","last_deep_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.826203+0000","objects_scrubbed":179,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:48:53.989588+0000","scrub_duration":0.225502547,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"53'108","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.907067+0000","last_change":"2023-10-30T12:57:56.907176+0000","last_active":"2023-10-30T12:57:56.907067+0000","last_peered":"2023-10-30T12:57:56.907067+0000","last_clean":"2023-10-30T12:57:56.907067+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:56.907067+0000","last_undegraded":"2023-10-30T12:57:56.907067+0000","last_fullsized":"2023-10-30T12:57:56.907067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'108","last_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_deep_scrub":"53'108","last_deep_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.906852+0000","objects_scrubbed":4,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:54.254936+0000","scrub_duration":0.023033504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":7391,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19720,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'357","reported_seq":2495,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668696+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:39.668696+0000","last_peered":"2023-10-30T12:57:39.668696+0000","last_clean":"2023-10-30T12:57:39.668696+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:39.668696+0000","last_undegraded":"2023-10-30T12:57:39.668696+0000","last_fullsized":"2023-10-30T12:57:39.668696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'404","reported_seq":472,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.447273+0000","last_change":"2023-10-30T12:58:00.447405+0000","last_active":"2023-10-30T12:58:00.447273+0000","last_peered":"2023-10-30T12:58:00.447273+0000","last_clean":"2023-10-30T12:58:00.447273+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:58:00.447273+0000","last_undegraded":"2023-10-30T12:58:00.447273+0000","last_fullsized":"2023-10-30T12:58:00.447273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'404","last_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_deep_scrub":"53'404","last_deep_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.447060+0000","objects_scrubbed":6,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:18.599324+0000","scrub_duration":0.056206357999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":392,"num_write_kb":16593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'331","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:40.038754+0000","last_change":"2023-10-30T12:52:38.959038+0000","last_active":"2023-10-30T12:57:40.038754+0000","last_peered":"2023-10-30T12:57:40.038754+0000","last_clean":"2023-10-30T12:57:40.038754+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:57:40.038754+0000","last_undegraded":"2023-10-30T12:57:40.038754+0000","last_fullsized":"2023-10-30T12:57:40.038754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"53'125","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.181850+0000","last_change":"2023-10-30T12:57:44.181906+0000","last_active":"2023-10-30T12:57:44.181850+0000","last_peered":"2023-10-30T12:57:44.181850+0000","last_clean":"2023-10-30T12:57:44.181850+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:44.181850+0000","last_undegraded":"2023-10-30T12:57:44.181850+0000","last_fullsized":"2023-10-30T12:57:44.181850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.181771+0000","objects_scrubbed":13,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:04:46.051796+0000","scrub_duration":0.017190094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51810,"num_omap_keys":95,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'357","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.753366+0000","last_change":"2023-10-30T12:52:38.707521+0000","last_active":"2023-10-30T12:57:39.753366+0000","last_peered":"2023-10-30T12:57:39.753366+0000","last_clean":"2023-10-30T12:57:39.753366+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:57:39.753366+0000","last_undegraded":"2023-10-30T12:57:39.753366+0000","last_fullsized":"2023-10-30T12:57:39.753366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'107","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.532064+0000","last_change":"2023-10-30T12:57:48.532178+0000","last_active":"2023-10-30T12:57:48.532064+0000","last_peered":"2023-10-30T12:57:48.532064+0000","last_clean":"2023-10-30T12:57:48.532064+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:48.532064+0000","last_undegraded":"2023-10-30T12:57:48.532064+0000","last_fullsized":"2023-10-30T12:57:48.532064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'107","last_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_deep_scrub":"53'107","last_deep_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.531861+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:05.413034+0000","scrub_duration":0.032263541999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":179,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'467","reported_seq":2603,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.047339+0000","last_change":"2023-10-30T12:58:00.047393+0000","last_active":"2023-10-30T12:58:00.047339+0000","last_peered":"2023-10-30T12:58:00.047339+0000","last_clean":"2023-10-30T12:58:00.047339+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:58:00.047339+0000","last_undegraded":"2023-10-30T12:58:00.047339+0000","last_fullsized":"2023-10-30T12:58:00.047339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'467","last_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_deep_scrub":"53'467","last_deep_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.047244+0000","objects_scrubbed":156,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:00.326328+0000","scrub_duration":0.23210165099999999,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12072627,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":307,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'170","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:59.087103+0000","last_change":"2023-10-30T12:57:59.087197+0000","last_active":"2023-10-30T12:57:59.087103+0000","last_peered":"2023-10-30T12:57:59.087103+0000","last_clean":"2023-10-30T12:57:59.087103+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:57:59.087103+0000","last_undegraded":"2023-10-30T12:57:59.087103+0000","last_fullsized":"2023-10-30T12:57:59.087103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'170","last_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_deep_scrub":"53'170","last_deep_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_clean_scrub_stamp":"2023-10-30T12:57:59.087009+0000","objects_scrubbed":5,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:31.789859+0000","scrub_duration":0.011509221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32668,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'339","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678777+0000","last_change":"2023-10-30T12:52:38.167701+0000","last_active":"2023-10-30T12:57:39.678777+0000","last_peered":"2023-10-30T12:57:39.678777+0000","last_clean":"2023-10-30T12:57:39.678777+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:57:39.678777+0000","last_undegraded":"2023-10-30T12:57:39.678777+0000","last_fullsized":"2023-10-30T12:57:39.678777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"52'123","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.239970+0000","last_change":"2023-10-30T12:45:18.101016+0000","last_active":"2023-10-30T12:57:07.239970+0000","last_peered":"2023-10-30T12:57:07.239970+0000","last_clean":"2023-10-30T12:57:07.239970+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:57:07.239970+0000","last_undegraded":"2023-10-30T12:57:07.239970+0000","last_fullsized":"2023-10-30T12:57:07.239970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'367","reported_seq":2488,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.676425+0000","last_change":"2023-10-30T12:52:38.194662+0000","last_active":"2023-10-30T12:57:39.676425+0000","last_peered":"2023-10-30T12:57:39.676425+0000","last_clean":"2023-10-30T12:57:39.676425+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:57:39.676425+0000","last_undegraded":"2023-10-30T12:57:39.676425+0000","last_fullsized":"2023-10-30T12:57:39.676425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526751,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":358,"num_write_kb":14373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"52'12","reported_seq":71,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:07.609849+0000","last_change":"2023-10-30T12:45:18.099328+0000","last_active":"2023-10-30T12:57:07.609849+0000","last_peered":"2023-10-30T12:57:07.609849+0000","last_clean":"2023-10-30T12:57:07.609849+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:57:07.609849+0000","last_undegraded":"2023-10-30T12:57:07.609849+0000","last_fullsized":"2023-10-30T12:57:07.609849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'333","reported_seq":2452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678290+0000","last_change":"2023-10-30T12:52:38.166263+0000","last_active":"2023-10-30T12:57:39.678290+0000","last_peered":"2023-10-30T12:57:39.678290+0000","last_clean":"2023-10-30T12:57:39.678290+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:57:39.678290+0000","last_undegraded":"2023-10-30T12:57:39.678290+0000","last_fullsized":"2023-10-30T12:57:39.678290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468449,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":323,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.811840+0000","last_change":"2023-10-30T12:57:48.811953+0000","last_active":"2023-10-30T12:57:48.811840+0000","last_peered":"2023-10-30T12:57:48.811840+0000","last_clean":"2023-10-30T12:57:48.811840+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:48.811840+0000","last_undegraded":"2023-10-30T12:57:48.811840+0000","last_fullsized":"2023-10-30T12:57:48.811840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.811641+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:27.857700+0000","scrub_duration":0.0086407700000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'15","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.871134+0000","last_change":"2023-10-30T12:57:49.871261+0000","last_active":"2023-10-30T12:57:49.871134+0000","last_peered":"2023-10-30T12:57:49.871134+0000","last_clean":"2023-10-30T12:57:49.871134+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:49.871134+0000","last_undegraded":"2023-10-30T12:57:49.871134+0000","last_fullsized":"2023-10-30T12:57:49.871134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.870906+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:10:34.313354+0000","scrub_duration":0.010713916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15500,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'310","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.700537+0000","last_change":"2023-10-30T12:52:38.374620+0000","last_active":"2023-10-30T12:57:44.700537+0000","last_peered":"2023-10-30T12:57:44.700537+0000","last_clean":"2023-10-30T12:57:44.700537+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:57:44.700537+0000","last_undegraded":"2023-10-30T12:57:44.700537+0000","last_fullsized":"2023-10-30T12:57:44.700537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"53'505","reported_seq":574,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.786677+0000","last_change":"2023-10-30T12:57:50.786811+0000","last_active":"2023-10-30T12:57:50.786677+0000","last_peered":"2023-10-30T12:57:50.786677+0000","last_clean":"2023-10-30T12:57:50.786677+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:50.786677+0000","last_undegraded":"2023-10-30T12:57:50.786677+0000","last_fullsized":"2023-10-30T12:57:50.786677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.786437+0000","objects_scrubbed":8,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:51.266608+0000","scrub_duration":0.043680202000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":44444,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'327","reported_seq":2436,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.081537+0000","last_change":"2023-10-30T12:52:38.159894+0000","last_active":"2023-10-30T12:57:43.081537+0000","last_peered":"2023-10-30T12:57:43.081537+0000","last_clean":"2023-10-30T12:57:43.081537+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:57:43.081537+0000","last_undegraded":"2023-10-30T12:57:43.081537+0000","last_fullsized":"2023-10-30T12:57:43.081537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'158","reported_seq":283,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:59.886690+0000","last_change":"2023-10-30T12:57:47.115495+0000","last_active":"2023-10-30T12:57:59.886690+0000","last_peered":"2023-10-30T12:57:59.886690+0000","last_clean":"2023-10-30T12:57:59.886690+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:57:59.886690+0000","last_undegraded":"2023-10-30T12:57:59.886690+0000","last_fullsized":"2023-10-30T12:57:59.886690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_clean_scrub_stamp":"2023-10-30T12:57:47.115321+0000","objects_scrubbed":9,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:48:57.832139+0000","scrub_duration":0.015406534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4547904,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":147,"num_write_kb":4595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63882,"num_omap_keys":117,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'325","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.617538+0000","last_change":"2023-10-30T12:52:38.157754+0000","last_active":"2023-10-30T12:57:39.617538+0000","last_peered":"2023-10-30T12:57:39.617538+0000","last_clean":"2023-10-30T12:57:39.617538+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:57:39.617538+0000","last_undegraded":"2023-10-30T12:57:39.617538+0000","last_fullsized":"2023-10-30T12:57:39.617538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"53'249","reported_seq":311,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.994818+0000","last_change":"2023-10-30T12:58:00.994936+0000","last_active":"2023-10-30T12:58:00.994818+0000","last_peered":"2023-10-30T12:58:00.994818+0000","last_clean":"2023-10-30T12:58:00.994818+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:58:00.994818+0000","last_undegraded":"2023-10-30T12:58:00.994818+0000","last_fullsized":"2023-10-30T12:58:00.994818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'249","last_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_deep_scrub":"53'249","last_deep_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.994611+0000","objects_scrubbed":4,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:45.189479+0000","scrub_duration":0.037553993000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21832,"num_omap_keys":40,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'541","reported_seq":2671,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.766867+0000","last_change":"2023-10-30T12:57:52.767026+0000","last_active":"2023-10-30T12:57:52.766867+0000","last_peered":"2023-10-30T12:57:52.766867+0000","last_clean":"2023-10-30T12:57:52.766867+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:52.766867+0000","last_undegraded":"2023-10-30T12:57:52.766867+0000","last_fullsized":"2023-10-30T12:57:52.766867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'541","last_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_deep_scrub":"53'541","last_deep_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.766665+0000","objects_scrubbed":172,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:47:06.068606+0000","scrub_duration":0.32093353800000002,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'3","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.619043+0000","last_change":"2023-10-30T12:45:18.081579+0000","last_active":"2023-10-30T12:57:43.619043+0000","last_peered":"2023-10-30T12:57:43.619043+0000","last_clean":"2023-10-30T12:57:43.619043+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:57:43.619043+0000","last_undegraded":"2023-10-30T12:57:43.619043+0000","last_fullsized":"2023-10-30T12:57:43.619043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'330","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797314+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:45.797314+0000","last_peered":"2023-10-30T12:57:45.797314+0000","last_clean":"2023-10-30T12:57:45.797314+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:45.797314+0000","last_undegraded":"2023-10-30T12:57:45.797314+0000","last_fullsized":"2023-10-30T12:57:45.797314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9698880,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":313,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'15","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.813043+0000","last_change":"2023-10-30T12:57:58.813159+0000","last_active":"2023-10-30T12:57:58.813043+0000","last_peered":"2023-10-30T12:57:58.813043+0000","last_clean":"2023-10-30T12:57:58.813043+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:58.813043+0000","last_undegraded":"2023-10-30T12:57:58.813043+0000","last_fullsized":"2023-10-30T12:57:58.813043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.812837+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:16.476751+0000","scrub_duration":0.009639435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25372,"num_omap_keys":48,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'341","reported_seq":2463,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.619111+0000","last_change":"2023-10-30T12:52:38.158164+0000","last_active":"2023-10-30T12:57:43.619111+0000","last_peered":"2023-10-30T12:57:43.619111+0000","last_clean":"2023-10-30T12:57:43.619111+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:57:43.619111+0000","last_undegraded":"2023-10-30T12:57:43.619111+0000","last_fullsized":"2023-10-30T12:57:43.619111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'100","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:41.406486+0000","last_change":"2023-10-30T12:45:18.085920+0000","last_active":"2023-10-30T12:57:41.406486+0000","last_peered":"2023-10-30T12:57:41.406486+0000","last_clean":"2023-10-30T12:57:41.406486+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:57:41.406486+0000","last_undegraded":"2023-10-30T12:57:41.406486+0000","last_fullsized":"2023-10-30T12:57:41.406486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":240,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'314","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672296+0000","last_change":"2023-10-30T12:52:38.374909+0000","last_active":"2023-10-30T12:57:39.672296+0000","last_peered":"2023-10-30T12:57:39.672296+0000","last_clean":"2023-10-30T12:57:39.672296+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:57:39.672296+0000","last_undegraded":"2023-10-30T12:57:39.672296+0000","last_fullsized":"2023-10-30T12:57:39.672296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"53'16","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:51.452606+0000","last_change":"2023-10-30T12:57:51.452812+0000","last_active":"2023-10-30T12:57:51.452606+0000","last_peered":"2023-10-30T12:57:51.452606+0000","last_clean":"2023-10-30T12:57:51.452606+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:51.452606+0000","last_undegraded":"2023-10-30T12:57:51.452606+0000","last_fullsized":"2023-10-30T12:57:51.452606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_clean_scrub_stamp":"2023-10-30T12:57:51.452333+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:18:15.321071+0000","scrub_duration":0.011134068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4360,"num_omap_keys":8,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'484","reported_seq":2621,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.782578+0000","last_change":"2023-10-30T12:57:48.782635+0000","last_active":"2023-10-30T12:57:48.782578+0000","last_peered":"2023-10-30T12:57:48.782578+0000","last_clean":"2023-10-30T12:57:48.782578+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:48.782578+0000","last_undegraded":"2023-10-30T12:57:48.782578+0000","last_fullsized":"2023-10-30T12:57:48.782578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.782504+0000","objects_scrubbed":167,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:48.560996+0000","scrub_duration":0.129876255,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'167","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.808055+0000","last_change":"2023-10-30T12:57:49.808170+0000","last_active":"2023-10-30T12:57:49.808055+0000","last_peered":"2023-10-30T12:57:49.808055+0000","last_clean":"2023-10-30T12:57:49.808055+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:49.808055+0000","last_undegraded":"2023-10-30T12:57:49.808055+0000","last_fullsized":"2023-10-30T12:57:49.808055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'167","last_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_deep_scrub":"53'167","last_deep_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.807844+0000","objects_scrubbed":4,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:19.047233+0000","scrub_duration":0.025125933999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6708,"num_omap_keys":13,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'495","reported_seq":2619,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:54.741282+0000","last_change":"2023-10-30T12:57:54.741397+0000","last_active":"2023-10-30T12:57:54.741282+0000","last_peered":"2023-10-30T12:57:54.741282+0000","last_clean":"2023-10-30T12:57:54.741282+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:54.741282+0000","last_undegraded":"2023-10-30T12:57:54.741282+0000","last_fullsized":"2023-10-30T12:57:54.741282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_clean_scrub_stamp":"2023-10-30T12:57:54.741074+0000","objects_scrubbed":156,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:22.048659+0000","scrub_duration":0.30326803000000002,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"52'58","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:35.418607+0000","last_change":"2023-10-30T12:45:18.113898+0000","last_active":"2023-10-30T12:57:35.418607+0000","last_peered":"2023-10-30T12:57:35.418607+0000","last_clean":"2023-10-30T12:57:35.418607+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:57:35.418607+0000","last_undegraded":"2023-10-30T12:57:35.418607+0000","last_fullsized":"2023-10-30T12:57:35.418607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:17.038065+0000","last_clean_scrub_stamp":"2023-10-30T12:45:17.038065+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'327","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.603249+0000","last_change":"2023-10-30T12:52:38.707846+0000","last_active":"2023-10-30T12:57:39.603249+0000","last_peered":"2023-10-30T12:57:39.603249+0000","last_clean":"2023-10-30T12:57:39.603249+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:57:39.603249+0000","last_undegraded":"2023-10-30T12:57:39.603249+0000","last_fullsized":"2023-10-30T12:57:39.603249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'122","reported_seq":183,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.127340+0000","last_change":"2023-10-30T12:58:00.127400+0000","last_active":"2023-10-30T12:58:00.127340+0000","last_peered":"2023-10-30T12:58:00.127340+0000","last_clean":"2023-10-30T12:58:00.127340+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:58:00.127340+0000","last_undegraded":"2023-10-30T12:58:00.127340+0000","last_fullsized":"2023-10-30T12:58:00.127340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'122","last_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_deep_scrub":"53'122","last_deep_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.127231+0000","objects_scrubbed":4,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:56.254642+0000","scrub_duration":0.026565129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":199,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18104,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'330","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.607157+0000","last_change":"2023-10-30T12:52:38.373796+0000","last_active":"2023-10-30T12:57:39.607157+0000","last_peered":"2023-10-30T12:57:39.607157+0000","last_clean":"2023-10-30T12:57:39.607157+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:57:39.607157+0000","last_undegraded":"2023-10-30T12:57:39.607157+0000","last_fullsized":"2023-10-30T12:57:39.607157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"53'192","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.821533+0000","last_change":"2023-10-30T12:57:56.821734+0000","last_active":"2023-10-30T12:57:56.821533+0000","last_peered":"2023-10-30T12:57:56.821533+0000","last_clean":"2023-10-30T12:57:56.821533+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:57:56.821533+0000","last_undegraded":"2023-10-30T12:57:56.821533+0000","last_fullsized":"2023-10-30T12:57:56.821533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'192","last_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_deep_scrub":"53'192","last_deep_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.821278+0000","objects_scrubbed":5,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:54:26.335029+0000","scrub_duration":0.022744626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":4107,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'488","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.602319+0000","last_change":"2023-10-30T12:58:01.602376+0000","last_active":"2023-10-30T12:58:01.602319+0000","last_peered":"2023-10-30T12:58:01.602319+0000","last_clean":"2023-10-30T12:58:01.602319+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:58:01.602319+0000","last_undegraded":"2023-10-30T12:58:01.602319+0000","last_fullsized":"2023-10-30T12:58:01.602319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'488","last_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_deep_scrub":"53'488","last_deep_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.602238+0000","objects_scrubbed":157,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:01.459702+0000","scrub_duration":0.22063908099999999,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'334","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797779+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:45.797779+0000","last_peered":"2023-10-30T12:57:45.797779+0000","last_clean":"2023-10-30T12:57:45.797779+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:45.797779+0000","last_undegraded":"2023-10-30T12:57:45.797779+0000","last_fullsized":"2023-10-30T12:57:45.797779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22516204,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":313,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"53'154","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.745224+0000","last_change":"2023-10-30T12:57:53.745301+0000","last_active":"2023-10-30T12:57:53.745224+0000","last_peered":"2023-10-30T12:57:53.745224+0000","last_clean":"2023-10-30T12:57:53.745224+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:57:53.745224+0000","last_undegraded":"2023-10-30T12:57:53.745224+0000","last_fullsized":"2023-10-30T12:57:53.745224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.745078+0000","objects_scrubbed":3,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:28:27.394467+0000","scrub_duration":0.016168252000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'335","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797670+0000","last_change":"2023-10-30T12:52:38.194210+0000","last_active":"2023-10-30T12:57:45.797670+0000","last_peered":"2023-10-30T12:57:45.797670+0000","last_clean":"2023-10-30T12:57:45.797670+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:57:45.797670+0000","last_undegraded":"2023-10-30T12:57:45.797670+0000","last_fullsized":"2023-10-30T12:57:45.797670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242846,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":314,"num_write_kb":14255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"53'369","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.867694+0000","last_change":"2023-10-30T12:57:55.867838+0000","last_active":"2023-10-30T12:57:55.867694+0000","last_peered":"2023-10-30T12:57:55.867694+0000","last_clean":"2023-10-30T12:57:55.867694+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:57:55.867694+0000","last_undegraded":"2023-10-30T12:57:55.867694+0000","last_fullsized":"2023-10-30T12:57:55.867694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'369","last_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_deep_scrub":"53'369","last_deep_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.867484+0000","objects_scrubbed":7,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:44.299714+0000","scrub_duration":0.065173216000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":22,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'335","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.081598+0000","last_change":"2023-10-30T12:52:38.162371+0000","last_active":"2023-10-30T12:57:43.081598+0000","last_peered":"2023-10-30T12:57:43.081598+0000","last_clean":"2023-10-30T12:57:43.081598+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:57:43.081598+0000","last_undegraded":"2023-10-30T12:57:43.081598+0000","last_fullsized":"2023-10-30T12:57:43.081598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"53'281","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.503383+0000","last_change":"2023-10-30T12:57:53.503441+0000","last_active":"2023-10-30T12:57:53.503383+0000","last_peered":"2023-10-30T12:57:53.503383+0000","last_clean":"2023-10-30T12:57:53.503383+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:53.503383+0000","last_undegraded":"2023-10-30T12:57:53.503383+0000","last_fullsized":"2023-10-30T12:57:53.503383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'281","last_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_deep_scrub":"53'281","last_deep_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.503283+0000","objects_scrubbed":3,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:22.524008+0000","scrub_duration":0.021426580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8098,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'331","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.638389+0000","last_change":"2023-10-30T12:52:38.373954+0000","last_active":"2023-10-30T12:57:39.638389+0000","last_peered":"2023-10-30T12:57:39.638389+0000","last_clean":"2023-10-30T12:57:39.638389+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:57:39.638389+0000","last_undegraded":"2023-10-30T12:57:39.638389+0000","last_fullsized":"2023-10-30T12:57:39.638389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"53'201","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.639282+0000","last_change":"2023-10-30T12:57:43.639282+0000","last_active":"2023-10-30T12:57:43.639282+0000","last_peered":"2023-10-30T12:57:43.639282+0000","last_clean":"2023-10-30T12:57:43.639282+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:57:43.639282+0000","last_undegraded":"2023-10-30T12:57:43.639282+0000","last_fullsized":"2023-10-30T12:57:43.639282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'201","last_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_deep_scrub":"53'201","last_deep_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_clean_scrub_stamp":"2023-10-30T12:57:43.639154+0000","objects_scrubbed":8,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:04.340919+0000","scrub_duration":0.033784741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925184388,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20887,"num_write_kb":1047691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902518272,"data_stored":1848828680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25189,"ondisk_log_size":25189,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":177519677,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":601,"num_read_kb":143265,"num_write":5675,"num_write_kb":179129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":556264,"num_omap_keys":1039,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":355188736,"data_stored":355039354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5547,"ondisk_log_size":5547,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":390676,"kb_used_data":327564,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":93981164,"statfs":{"total":96636764160,"available":96236711936,"internally_reserved":0,"allocated":335425536,"data_stored":327567493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":64616907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":0.95699999999999996}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.103}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":1.482}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":1.425},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.2430000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.9019999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":2.097}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287404,"kb_used_data":238180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94084436,"statfs":{"total":96636764160,"available":96342462464,"internally_reserved":0,"allocated":243896320,"data_stored":238678405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.028}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":1.101}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.875}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.82899999999999996}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.2470000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":1.298},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.3600000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.4019999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365204,"kb_used_data":293772,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94006636,"statfs":{"total":96636764160,"available":96262795264,"internally_reserved":0,"allocated":300822528,"data_stored":293054021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":1.081},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":1.1910000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.286},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":0.84399999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":1.339},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":0.874},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.387}]}]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405148,"kb_used_data":342036,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":93966692,"statfs":{"total":96636764160,"available":96221892608,"internally_reserved":0,"allocated":350244864,"data_stored":342085196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":1.024}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.96399999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.0469999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.997}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":1.1359999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410532,"kb_used_data":334940,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93961308,"statfs":{"total":96636764160,"available":96216379392,"internally_reserved":0,"allocated":342978560,"data_stored":334674863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77396424},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.436},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.47899999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Oct 30 12:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.499},"max":{"1min":0.98099999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83799999999999997},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.226,"5min":1.786,"15min":1.867},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":1.1100000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.753},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.97299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":177888,"kb_used_data":150872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94193952,"statfs":{"total":96636764160,"available":96454606848,"internally_reserved":0,"allocated":154492928,"data_stored":149417131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.78000000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.91200000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266344,"kb_used_data":212960,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94105496,"statfs":{"total":96636764160,"available":96364027904,"internally_reserved":0,"allocated":218071040,"data_stored":212483944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.63},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.65100000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.995}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":1.0369999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":1.0149999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373504,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":93998336,"statfs":{"total":96636764160,"available":96254296064,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.377}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.68799999999999994}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.61499999999999999}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12959744,"data_stored":12937240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42500096,"data_stored":42479449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61378560,"data_stored":61363749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35946496,"data_stored":35926937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":51351552,"data_stored":51334703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264265728,"data_stored":256824371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":262901760,"data_stored":255370516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:58:07.775 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.775 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.776 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.776 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.777 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.777 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.777 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.778 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.778 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.779 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.779 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.779 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.780 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.780 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.781 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.781 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.781 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.782 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.782 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.783 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.783 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.783 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.784 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.784 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.785 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.785 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.786 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.786 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.786 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.787 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.787 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.787 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.788 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.788 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.789 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.789 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.790 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.790 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.790 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.791 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.791 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.791 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2023-10-30T12:45:17.038065+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.792 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.792 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.792 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.793 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.793 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.794 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:07.794 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:58:16.346 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:58:16.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:58:27.795 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:58:27.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 Processor -- start 2023-10-30T12:58:27.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 -- start start 2023-10-30T12:58:27.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1cc15ac70 0x7fe1cc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc15dcc0 con 0x7fe1cc06aeb0 2023-10-30T12:58:27.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc159800 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d23a7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc159d80 con 0x7fe1cc15ac70 2023-10-30T12:58:27.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44064/0 (socket says 172.21.15.8:44064) 2023-10-30T12:58:27.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 -- 172.21.15.8:0/85819530 learned_addr learned my addr 172.21.15.8:0/85819530 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:27.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d0ba4640 1 --2- 172.21.15.8:0/85819530 >> v2:172.21.15.46:3300/0 conn(0x7fe1cc15ac70 0x7fe1cc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 -- 172.21.15.8:0/85819530 >> v2:172.21.15.46:3300/0 conn(0x7fe1cc15ac70 msgr2=0x7fe1cc15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/85819530 >> v2:172.21.15.46:3300/0 conn(0x7fe1cc15ac70 0x7fe1cc15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 -- 172.21.15.8:0/85819530 >> v1:172.21.15.46:6789/0 conn(0x7fe1cc06aeb0 legacy=0x7fe1cc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.946+0000 7fe1d13a5640 1 -- 172.21.15.8:0/85819530 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc15a750 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/85819530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b4130 tx=0x7fe1c801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50344ad7f196984 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:27.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1c3fff640 1 -- 172.21.15.8:0/85819530 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a89b0 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1c3fff640 1 -- 172.21.15.8:0/85819530 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c80a8b10 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1c3fff640 1 -- 172.21.15.8:0/85819530 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a8ea0 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 -- 172.21.15.8:0/85819530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc0c8eb0 msgr2=0x7fe1cc15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:27.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/85819530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b4130 tx=0x7fe1c801ecb0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 -- 172.21.15.8:0/85819530 shutdown_connections 2023-10-30T12:58:27.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/85819530 >> v2:172.21.15.46:3300/0 conn(0x7fe1cc15ac70 0x7fe1cc15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/85819530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc15cd70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 -- 172.21.15.8:0/85819530 >> 172.21.15.8:0/85819530 conn(0x7fe1cc0cef00 msgr2=0x7fe1cc0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:27.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 -- 172.21.15.8:0/85819530 shutdown_connections 2023-10-30T12:58:27.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 -- 172.21.15.8:0/85819530 wait complete. 2023-10-30T12:58:27.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.950+0000 7fe1d23a7640 1 Processor -- start 2023-10-30T12:58:27.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- start start 2023-10-30T12:58:27.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 0x7fe1cc0c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0c3870 con 0x7fe1cc15ac70 2023-10-30T12:58:27.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15ca30 con 0x7fe1cc0c8eb0 2023-10-30T12:58:27.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1568f0 con 0x7fe1cc06aeb0 2023-10-30T12:58:27.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44066/0 (socket says 172.21.15.8:44066) 2023-10-30T12:58:27.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 -- 172.21.15.8:0/3851312498 learned_addr learned my addr 172.21.15.8:0/3851312498 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:27.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d0ba4640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 0x7fe1cc0c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 -- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 msgr2=0x7fe1cc0c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:27.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 0x7fe1cc0c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 -- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 msgr2=0x7fe1cc0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:27.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:27.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 -- 172.21.15.8:0/3851312498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80094d0 con 0x7fe1cc15ac70 2023-10-30T12:58:27.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 0x7fe1cc0c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:27.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d0ba4640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:27.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d1ba6640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b970 tx=0x7fe1c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:27.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400cb40 con 0x7fe1cc15ac70 2023-10-30T12:58:27.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c400cca0 con 0x7fe1cc15ac70 2023-10-30T12:58:27.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0c55f0 con 0x7fe1cc15ac70 2023-10-30T12:58:27.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4014680 con 0x7fe1cc15ac70 2023-10-30T12:58:27.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc156e40 con 0x7fe1cc15ac70 2023-10-30T12:58:27.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.954+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc154670 con 0x7fe1cc15ac70 2023-10-30T12:58:27.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.958+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe1c401d3b0 con 0x7fe1cc15ac70 2023-10-30T12:58:27.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.958+0000 7fe1c1ffb640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 0x7fe1b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:27.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.958+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7fe1c4083fe0 con 0x7fe1cc15ac70 2023-10-30T12:58:27.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.958+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 0x7fe1b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:27.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.958+0000 7fe1d13a5640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 0x7fe1b0068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80b2000 tx=0x7fe1c8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:27.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:27.962+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe1cc154670 con 0x7fe1cc15ac70 2023-10-30T12:58:28.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.118+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1cc15d700 con 0x7fe1b00664a0 2023-10-30T12:58:28.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.126+0000 7fe1c1ffb640 1 -- 172.21.15.8:0/3851312498 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294250 (secure 0 0 0) 0x7fe1c80bb420 con 0x7fe1b00664a0 2023-10-30T12:58:28.133 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:28.135 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:58:28.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 msgr2=0x7fe1b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:28.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 0x7fe1b0068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80b2000 tx=0x7fe1c8008040 comp rx=0 tx=0).stop 2023-10-30T12:58:28.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 msgr2=0x7fe1cc154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:28.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b970 tx=0x7fe1c400be30 comp rx=0 tx=0).stop 2023-10-30T12:58:28.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 shutdown_connections 2023-10-30T12:58:28.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7fe1b00664a0 0x7fe1b0068960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:28.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1cc0c8eb0 0x7fe1cc0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:28.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1cc15ac70 0x7fe1cc154950 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:28.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 --2- 172.21.15.8:0/3851312498 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7fe1cc06aeb0 0x7fe1cc0c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:28.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 >> 172.21.15.8:0/3851312498 conn(0x7fe1cc0cef00 msgr2=0x7fe1cc0c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:28.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 shutdown_connections 2023-10-30T12:58:28.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:28.130+0000 7fe1d23a7640 1 -- 172.21.15.8:0/3851312498 wait complete. 2023-10-30T12:58:28.153 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2023-10-30T12:58:27.819198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1103521598,"num_objects":10479,"num_object_clones":0,"num_object_copies":20958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10479,"num_whiteouts":0,"num_read":2548,"num_read_kb":243548,"num_write":26744,"num_write_kb":1229467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":692939,"num_omap_keys":1293,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36734,"ondisk_log_size":36734,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2728484,"kb_used_data":2211172,"kb_used_omap":12,"kb_used_meta":517235,"kb_avail":752246236,"statfs":{"total":773094113280,"available":770300145664,"internally_reserved":0,"allocated":2264240128,"data_stored":2207923785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":529649210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":111928,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002943"},"pg_stats":[{"pgid":"3.3f","version":"53'504","reported_seq":2639,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.025636+0000","last_change":"2023-10-30T12:58:07.025697+0000","last_active":"2023-10-30T12:58:07.025636+0000","last_peered":"2023-10-30T12:58:07.025636+0000","last_clean":"2023-10-30T12:58:07.025636+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:58:07.025636+0000","last_undegraded":"2023-10-30T12:58:07.025636+0000","last_fullsized":"2023-10-30T12:58:07.025636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'504","last_scrub_stamp":"2023-10-30T12:58:07.025535+0000","last_deep_scrub":"53'504","last_deep_scrub_stamp":"2023-10-30T12:58:07.025535+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.025535+0000","objects_scrubbed":163,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:29:12.472868+0000","scrub_duration":0.24485915899999999,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'506","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:21.271626+0000","last_change":"2023-10-30T12:58:21.271719+0000","last_active":"2023-10-30T12:58:21.271626+0000","last_peered":"2023-10-30T12:58:21.271626+0000","last_clean":"2023-10-30T12:58:21.271626+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:58:21.271626+0000","last_undegraded":"2023-10-30T12:58:21.271626+0000","last_fullsized":"2023-10-30T12:58:21.271626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'506","last_scrub_stamp":"2023-10-30T12:58:21.271522+0000","last_deep_scrub":"53'506","last_deep_scrub_stamp":"2023-10-30T12:58:21.271522+0000","last_clean_scrub_stamp":"2023-10-30T12:58:21.271522+0000","objects_scrubbed":170,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:55:22.196209+0000","scrub_duration":0.153310585,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'510","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.591196+0000","last_change":"2023-10-30T12:57:58.591267+0000","last_active":"2023-10-30T12:57:58.591196+0000","last_peered":"2023-10-30T12:57:58.591196+0000","last_clean":"2023-10-30T12:57:58.591196+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:58.591196+0000","last_undegraded":"2023-10-30T12:57:58.591196+0000","last_fullsized":"2023-10-30T12:57:58.591196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'510","last_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_deep_scrub":"53'510","last_deep_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.591110+0000","objects_scrubbed":152,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:03:49.294530+0000","scrub_duration":0.16537424100000001,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'301","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.677349+0000","last_change":"2023-10-30T12:52:38.193926+0000","last_active":"2023-10-30T12:57:39.677349+0000","last_peered":"2023-10-30T12:57:39.677349+0000","last_clean":"2023-10-30T12:57:39.677349+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:57:39.677349+0000","last_undegraded":"2023-10-30T12:57:39.677349+0000","last_fullsized":"2023-10-30T12:57:39.677349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'507","reported_seq":2636,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.102205+0000","last_change":"2023-10-30T12:58:19.102288+0000","last_active":"2023-10-30T12:58:19.102205+0000","last_peered":"2023-10-30T12:58:19.102205+0000","last_clean":"2023-10-30T12:58:19.102205+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:58:19.102205+0000","last_undegraded":"2023-10-30T12:58:19.102205+0000","last_fullsized":"2023-10-30T12:58:19.102205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-30T12:58:19.102061+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-30T12:58:19.102061+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.102061+0000","objects_scrubbed":156,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:30:09.576623+0000","scrub_duration":0.33207663799999998,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28213093,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":332,"num_write_kb":27731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'487","reported_seq":2615,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:11.100664+0000","last_change":"2023-10-30T12:58:11.100708+0000","last_active":"2023-10-30T12:58:11.100664+0000","last_peered":"2023-10-30T12:58:11.100664+0000","last_clean":"2023-10-30T12:58:11.100664+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:58:11.100664+0000","last_undegraded":"2023-10-30T12:58:11.100664+0000","last_fullsized":"2023-10-30T12:58:11.100664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:11.100583+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:11.100583+0000","last_clean_scrub_stamp":"2023-10-30T12:58:11.100583+0000","objects_scrubbed":155,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:33:07.724909+0000","scrub_duration":0.13540437999999999,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'546","reported_seq":2669,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.122880+0000","last_change":"2023-10-30T12:57:48.122880+0000","last_active":"2023-10-30T12:57:48.122880+0000","last_peered":"2023-10-30T12:57:48.122880+0000","last_clean":"2023-10-30T12:57:48.122880+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:48.122880+0000","last_undegraded":"2023-10-30T12:57:48.122880+0000","last_fullsized":"2023-10-30T12:57:48.122880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'546","last_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_deep_scrub":"53'546","last_deep_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.122543+0000","objects_scrubbed":180,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:00.797869+0000","scrub_duration":0.33340951699999999,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23819931,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":356,"num_write_kb":23455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'449","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.579958+0000","last_change":"2023-10-30T12:58:03.580002+0000","last_active":"2023-10-30T12:58:03.579958+0000","last_peered":"2023-10-30T12:58:03.579958+0000","last_clean":"2023-10-30T12:58:03.579958+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:58:03.579958+0000","last_undegraded":"2023-10-30T12:58:03.579958+0000","last_fullsized":"2023-10-30T12:58:03.579958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.579888+0000","objects_scrubbed":134,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:08:01.622733+0000","scrub_duration":0.15696447399999999,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'496","reported_seq":2638,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.995994+0000","last_change":"2023-10-30T12:58:07.996083+0000","last_active":"2023-10-30T12:58:07.995994+0000","last_peered":"2023-10-30T12:58:07.995994+0000","last_clean":"2023-10-30T12:58:07.995994+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:58:07.995994+0000","last_undegraded":"2023-10-30T12:58:07.995994+0000","last_fullsized":"2023-10-30T12:58:07.995994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:07.995885+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:07.995885+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.995885+0000","objects_scrubbed":159,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:06.335910+0000","scrub_duration":0.243363578,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767488,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":327,"num_write_kb":17709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'484","reported_seq":2626,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.665230+0000","last_change":"2023-10-30T12:58:10.665308+0000","last_active":"2023-10-30T12:58:10.665230+0000","last_peered":"2023-10-30T12:58:10.665230+0000","last_clean":"2023-10-30T12:58:10.665230+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:58:10.665230+0000","last_undegraded":"2023-10-30T12:58:10.665230+0000","last_fullsized":"2023-10-30T12:58:10.665230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-30T12:58:10.665118+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-30T12:58:10.665118+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.665118+0000","objects_scrubbed":145,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:10:19.686235+0000","scrub_duration":0.26126349399999999,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'500","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.600687+0000","last_change":"2023-10-30T12:58:03.600771+0000","last_active":"2023-10-30T12:58:03.600687+0000","last_peered":"2023-10-30T12:58:03.600687+0000","last_clean":"2023-10-30T12:58:03.600687+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:58:03.600687+0000","last_undegraded":"2023-10-30T12:58:03.600687+0000","last_fullsized":"2023-10-30T12:58:03.600687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.600594+0000","objects_scrubbed":164,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:38.231319+0000","scrub_duration":0.225903303,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'459","reported_seq":2586,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.648328+0000","last_change":"2023-10-30T12:57:55.648414+0000","last_active":"2023-10-30T12:57:55.648328+0000","last_peered":"2023-10-30T12:57:55.648328+0000","last_clean":"2023-10-30T12:57:55.648328+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:55.648328+0000","last_undegraded":"2023-10-30T12:57:55.648328+0000","last_fullsized":"2023-10-30T12:57:55.648328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'459","last_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_deep_scrub":"53'459","last_deep_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.648231+0000","objects_scrubbed":147,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:45:01.167844+0000","scrub_duration":0.24832579699999999,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'477","reported_seq":2616,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:17.669645+0000","last_change":"2023-10-30T12:58:17.669698+0000","last_active":"2023-10-30T12:58:17.669645+0000","last_peered":"2023-10-30T12:58:17.669645+0000","last_clean":"2023-10-30T12:58:17.669645+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:58:17.669645+0000","last_undegraded":"2023-10-30T12:58:17.669645+0000","last_fullsized":"2023-10-30T12:58:17.669645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-30T12:58:17.669571+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-30T12:58:17.669571+0000","last_clean_scrub_stamp":"2023-10-30T12:58:17.669571+0000","objects_scrubbed":163,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:48.499603+0000","scrub_duration":0.21456085499999999,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'500","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.561433+0000","last_change":"2023-10-30T12:58:03.561531+0000","last_active":"2023-10-30T12:58:03.561433+0000","last_peered":"2023-10-30T12:58:03.561433+0000","last_clean":"2023-10-30T12:58:03.561433+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:58:03.561433+0000","last_undegraded":"2023-10-30T12:58:03.561433+0000","last_fullsized":"2023-10-30T12:58:03.561433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.561319+0000","objects_scrubbed":148,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:27.469036+0000","scrub_duration":0.14169902500000001,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'510","reported_seq":2641,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.347606+0000","last_change":"2023-10-30T12:58:19.347670+0000","last_active":"2023-10-30T12:58:19.347606+0000","last_peered":"2023-10-30T12:58:19.347606+0000","last_clean":"2023-10-30T12:58:19.347606+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:58:19.347606+0000","last_undegraded":"2023-10-30T12:58:19.347606+0000","last_fullsized":"2023-10-30T12:58:19.347606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'510","last_scrub_stamp":"2023-10-30T12:58:19.347521+0000","last_deep_scrub":"53'510","last_deep_scrub_stamp":"2023-10-30T12:58:19.347521+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.347521+0000","objects_scrubbed":166,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:18.461141+0000","scrub_duration":0.21695156800000001,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'535","reported_seq":2673,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.149981+0000","last_change":"2023-10-30T12:58:09.150069+0000","last_active":"2023-10-30T12:58:09.149981+0000","last_peered":"2023-10-30T12:58:09.149981+0000","last_clean":"2023-10-30T12:58:09.149981+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:58:09.149981+0000","last_undegraded":"2023-10-30T12:58:09.149981+0000","last_fullsized":"2023-10-30T12:58:09.149981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'535","last_scrub_stamp":"2023-10-30T12:58:09.149880+0000","last_deep_scrub":"53'535","last_deep_scrub_stamp":"2023-10-30T12:58:09.149880+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.149880+0000","objects_scrubbed":166,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:57:22.762278+0000","scrub_duration":0.178880715,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23758433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":359,"num_write_kb":26630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'338","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.678058+0000","last_change":"2023-10-30T12:52:38.196239+0000","last_active":"2023-10-30T12:57:39.678058+0000","last_peered":"2023-10-30T12:57:39.678058+0000","last_clean":"2023-10-30T12:57:39.678058+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:57:39.678058+0000","last_undegraded":"2023-10-30T12:57:39.678058+0000","last_fullsized":"2023-10-30T12:57:39.678058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10013359,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":323,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'508","reported_seq":2652,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:12.252082+0000","last_change":"2023-10-30T12:58:12.252141+0000","last_active":"2023-10-30T12:58:12.252082+0000","last_peered":"2023-10-30T12:58:12.252082+0000","last_clean":"2023-10-30T12:58:12.252082+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:58:12.252082+0000","last_undegraded":"2023-10-30T12:58:12.252082+0000","last_fullsized":"2023-10-30T12:58:12.252082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2023-10-30T12:58:12.252006+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2023-10-30T12:58:12.252006+0000","last_clean_scrub_stamp":"2023-10-30T12:58:12.252006+0000","objects_scrubbed":162,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:56:22.896888+0000","scrub_duration":0.22659649400000001,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'524","reported_seq":2651,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.142463+0000","last_change":"2023-10-30T12:58:22.142570+0000","last_active":"2023-10-30T12:58:22.142463+0000","last_peered":"2023-10-30T12:58:22.142463+0000","last_clean":"2023-10-30T12:58:22.142463+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:58:22.142463+0000","last_undegraded":"2023-10-30T12:58:22.142463+0000","last_fullsized":"2023-10-30T12:58:22.142463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'524","last_scrub_stamp":"2023-10-30T12:58:22.142304+0000","last_deep_scrub":"53'524","last_deep_scrub_stamp":"2023-10-30T12:58:22.142304+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.142304+0000","objects_scrubbed":163,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:29.904983+0000","scrub_duration":0.31798768799999999,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695113,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":341,"num_write_kb":18629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'507","reported_seq":2639,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.614493+0000","last_change":"2023-10-30T12:58:13.614550+0000","last_active":"2023-10-30T12:58:13.614493+0000","last_peered":"2023-10-30T12:58:13.614493+0000","last_clean":"2023-10-30T12:58:13.614493+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:58:13.614493+0000","last_undegraded":"2023-10-30T12:58:13.614493+0000","last_fullsized":"2023-10-30T12:58:13.614493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-30T12:58:13.614391+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-30T12:58:13.614391+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.614391+0000","objects_scrubbed":156,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:20.070473+0000","scrub_duration":0.212317802,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'517","reported_seq":2656,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:18.634707+0000","last_change":"2023-10-30T12:58:18.634758+0000","last_active":"2023-10-30T12:58:18.634707+0000","last_peered":"2023-10-30T12:58:18.634707+0000","last_clean":"2023-10-30T12:58:18.634707+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:58:18.634707+0000","last_undegraded":"2023-10-30T12:58:18.634707+0000","last_fullsized":"2023-10-30T12:58:18.634707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'517","last_scrub_stamp":"2023-10-30T12:58:18.634630+0000","last_deep_scrub":"53'517","last_deep_scrub_stamp":"2023-10-30T12:58:18.634630+0000","last_clean_scrub_stamp":"2023-10-30T12:58:18.634630+0000","objects_scrubbed":162,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:03.977601+0000","scrub_duration":0.23488492599999999,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'561","reported_seq":2700,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:16.293081+0000","last_change":"2023-10-30T12:58:16.293128+0000","last_active":"2023-10-30T12:58:16.293081+0000","last_peered":"2023-10-30T12:58:16.293081+0000","last_clean":"2023-10-30T12:58:16.293081+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:58:16.293081+0000","last_undegraded":"2023-10-30T12:58:16.293081+0000","last_fullsized":"2023-10-30T12:58:16.293081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'561","last_scrub_stamp":"2023-10-30T12:58:16.292999+0000","last_deep_scrub":"53'561","last_deep_scrub_stamp":"2023-10-30T12:58:16.292999+0000","last_clean_scrub_stamp":"2023-10-30T12:58:16.292999+0000","objects_scrubbed":178,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:58:07.559376+0000","scrub_duration":0.26458689699999999,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'533","reported_seq":2677,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.612587+0000","last_change":"2023-10-30T12:58:01.612683+0000","last_active":"2023-10-30T12:58:01.612587+0000","last_peered":"2023-10-30T12:58:01.612587+0000","last_clean":"2023-10-30T12:58:01.612587+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:58:01.612587+0000","last_undegraded":"2023-10-30T12:58:01.612587+0000","last_fullsized":"2023-10-30T12:58:01.612587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'533","last_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_deep_scrub":"53'533","last_deep_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.612496+0000","objects_scrubbed":161,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:32.894742+0000","scrub_duration":0.16518128200000001,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'534","reported_seq":2662,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.217312+0000","last_change":"2023-10-30T12:58:03.217312+0000","last_active":"2023-10-30T12:58:03.217312+0000","last_peered":"2023-10-30T12:58:03.217312+0000","last_clean":"2023-10-30T12:58:03.217312+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:58:03.217312+0000","last_undegraded":"2023-10-30T12:58:03.217312+0000","last_fullsized":"2023-10-30T12:58:03.217312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'534","last_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_deep_scrub":"53'534","last_deep_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.216977+0000","objects_scrubbed":173,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:50:07.436714+0000","scrub_duration":0.30228984199999998,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'495","reported_seq":2624,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:12.715377+0000","last_change":"2023-10-30T12:58:12.715377+0000","last_active":"2023-10-30T12:58:12.715377+0000","last_peered":"2023-10-30T12:58:12.715377+0000","last_clean":"2023-10-30T12:58:12.715377+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:58:12.715377+0000","last_undegraded":"2023-10-30T12:58:12.715377+0000","last_fullsized":"2023-10-30T12:58:12.715377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:58:12.715237+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:58:12.715237+0000","last_clean_scrub_stamp":"2023-10-30T12:58:12.715237+0000","objects_scrubbed":168,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:51:52.292380+0000","scrub_duration":0.228241844,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'545","reported_seq":2692,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:14.584433+0000","last_change":"2023-10-30T12:58:14.584433+0000","last_active":"2023-10-30T12:58:14.584433+0000","last_peered":"2023-10-30T12:58:14.584433+0000","last_clean":"2023-10-30T12:58:14.584433+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:58:14.584433+0000","last_undegraded":"2023-10-30T12:58:14.584433+0000","last_fullsized":"2023-10-30T12:58:14.584433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'545","last_scrub_stamp":"2023-10-30T12:58:14.584283+0000","last_deep_scrub":"53'545","last_deep_scrub_stamp":"2023-10-30T12:58:14.584283+0000","last_clean_scrub_stamp":"2023-10-30T12:58:14.584283+0000","objects_scrubbed":166,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:37:08.795274+0000","scrub_duration":0.223781852,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'328","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.619281+0000","last_change":"2023-10-30T12:52:38.158568+0000","last_active":"2023-10-30T12:57:43.619281+0000","last_peered":"2023-10-30T12:57:43.619281+0000","last_clean":"2023-10-30T12:57:43.619281+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:57:43.619281+0000","last_undegraded":"2023-10-30T12:57:43.619281+0000","last_fullsized":"2023-10-30T12:57:43.619281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'433","reported_seq":2561,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.668381+0000","last_change":"2023-10-30T12:58:22.668425+0000","last_active":"2023-10-30T12:58:22.668381+0000","last_peered":"2023-10-30T12:58:22.668381+0000","last_clean":"2023-10-30T12:58:22.668381+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:58:22.668381+0000","last_undegraded":"2023-10-30T12:58:22.668381+0000","last_fullsized":"2023-10-30T12:58:22.668381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'433","last_scrub_stamp":"2023-10-30T12:58:22.668305+0000","last_deep_scrub":"53'433","last_deep_scrub_stamp":"2023-10-30T12:58:22.668305+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.668305+0000","objects_scrubbed":127,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:21.611583+0000","scrub_duration":0.16211444699999999,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'481","reported_seq":2612,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.751923+0000","last_change":"2023-10-30T12:57:49.751923+0000","last_active":"2023-10-30T12:57:49.751923+0000","last_peered":"2023-10-30T12:57:49.751923+0000","last_clean":"2023-10-30T12:57:49.751923+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:49.751923+0000","last_undegraded":"2023-10-30T12:57:49.751923+0000","last_fullsized":"2023-10-30T12:57:49.751923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'481","last_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_deep_scrub":"53'481","last_deep_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.751798+0000","objects_scrubbed":156,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:28.001220+0000","scrub_duration":0.14818989499999999,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'555","reported_seq":2692,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.242980+0000","last_change":"2023-10-30T12:58:10.243033+0000","last_active":"2023-10-30T12:58:10.242980+0000","last_peered":"2023-10-30T12:58:10.242980+0000","last_clean":"2023-10-30T12:58:10.242980+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:58:10.242980+0000","last_undegraded":"2023-10-30T12:58:10.242980+0000","last_fullsized":"2023-10-30T12:58:10.242980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'555","last_scrub_stamp":"2023-10-30T12:58:10.242873+0000","last_deep_scrub":"53'555","last_deep_scrub_stamp":"2023-10-30T12:58:10.242873+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.242873+0000","objects_scrubbed":180,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:28:57.920616+0000","scrub_duration":0.241089517,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15318011,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":358,"num_write_kb":15146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'551","reported_seq":2689,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.240366+0000","last_change":"2023-10-30T12:58:10.240430+0000","last_active":"2023-10-30T12:58:10.240366+0000","last_peered":"2023-10-30T12:58:10.240366+0000","last_clean":"2023-10-30T12:58:10.240366+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:58:10.240366+0000","last_undegraded":"2023-10-30T12:58:10.240366+0000","last_fullsized":"2023-10-30T12:58:10.240366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'551","last_scrub_stamp":"2023-10-30T12:58:10.240254+0000","last_deep_scrub":"53'551","last_deep_scrub_stamp":"2023-10-30T12:58:10.240254+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.240254+0000","objects_scrubbed":175,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:53.225603+0000","scrub_duration":0.169709203,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'476","reported_seq":2608,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:04.621477+0000","last_change":"2023-10-30T12:58:04.621599+0000","last_active":"2023-10-30T12:58:04.621477+0000","last_peered":"2023-10-30T12:58:04.621477+0000","last_clean":"2023-10-30T12:58:04.621477+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:58:04.621477+0000","last_undegraded":"2023-10-30T12:58:04.621477+0000","last_fullsized":"2023-10-30T12:58:04.621477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_clean_scrub_stamp":"2023-10-30T12:58:04.621266+0000","objects_scrubbed":140,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:22:51.575535+0000","scrub_duration":0.236240279,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'492","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.717128+0000","last_change":"2023-10-30T12:57:50.717216+0000","last_active":"2023-10-30T12:57:50.717128+0000","last_peered":"2023-10-30T12:57:50.717128+0000","last_clean":"2023-10-30T12:57:50.717128+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:50.717128+0000","last_undegraded":"2023-10-30T12:57:50.717128+0000","last_fullsized":"2023-10-30T12:57:50.717128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'492","last_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_deep_scrub":"53'492","last_deep_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.717022+0000","objects_scrubbed":158,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:37.999028+0000","scrub_duration":0.26506830399999998,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"53'267","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.465321+0000","last_change":"2023-10-30T12:57:48.465321+0000","last_active":"2023-10-30T12:57:48.465321+0000","last_peered":"2023-10-30T12:57:48.465321+0000","last_clean":"2023-10-30T12:57:48.465321+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:48.465321+0000","last_undegraded":"2023-10-30T12:57:48.465321+0000","last_fullsized":"2023-10-30T12:57:48.465321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'267","last_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_deep_scrub":"53'267","last_deep_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.465192+0000","objects_scrubbed":5,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:16.586033+0000","scrub_duration":0.036637109000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8500,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'466","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:46.810452+0000","last_change":"2023-10-30T12:57:46.810452+0000","last_active":"2023-10-30T12:57:46.810452+0000","last_peered":"2023-10-30T12:57:46.810452+0000","last_clean":"2023-10-30T12:57:46.810452+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:46.810452+0000","last_undegraded":"2023-10-30T12:57:46.810452+0000","last_fullsized":"2023-10-30T12:57:46.810452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_clean_scrub_stamp":"2023-10-30T12:57:46.810303+0000","objects_scrubbed":147,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:25:30.111932+0000","scrub_duration":0.192780493,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'316","reported_seq":450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.292751+0000","last_change":"2023-10-30T12:57:57.478006+0000","last_active":"2023-10-30T12:58:15.292751+0000","last_peered":"2023-10-30T12:58:15.292751+0000","last_clean":"2023-10-30T12:58:15.292751+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:58:15.292751+0000","last_undegraded":"2023-10-30T12:58:15.292751+0000","last_fullsized":"2023-10-30T12:58:15.292751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'311","last_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_deep_scrub":"53'311","last_deep_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.477645+0000","objects_scrubbed":9,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:49:22.878603+0000","scrub_duration":0.042990041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11231324,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":305,"num_write_kb":11197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":49566,"num_omap_keys":93,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"53'74","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.463815+0000","last_change":"2023-10-30T12:57:49.463904+0000","last_active":"2023-10-30T12:57:49.463815+0000","last_peered":"2023-10-30T12:57:49.463815+0000","last_clean":"2023-10-30T12:57:49.463815+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:49.463815+0000","last_undegraded":"2023-10-30T12:57:49.463815+0000","last_fullsized":"2023-10-30T12:57:49.463815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'74","last_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_deep_scrub":"53'74","last_deep_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.463707+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:46:36.321809+0000","scrub_duration":0.0076797799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'574","reported_seq":2721,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.610591+0000","last_change":"2023-10-30T12:57:58.610634+0000","last_active":"2023-10-30T12:57:58.610591+0000","last_peered":"2023-10-30T12:57:58.610591+0000","last_clean":"2023-10-30T12:57:58.610591+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:58.610591+0000","last_undegraded":"2023-10-30T12:57:58.610591+0000","last_fullsized":"2023-10-30T12:57:58.610591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'574","last_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_deep_scrub":"53'574","last_deep_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.610507+0000","objects_scrubbed":176,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:34.361815+0000","scrub_duration":0.18830925700000001,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"53'347","reported_seq":407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.875242+0000","last_change":"2023-10-30T12:57:45.875338+0000","last_active":"2023-10-30T12:57:45.875242+0000","last_peered":"2023-10-30T12:57:45.875242+0000","last_clean":"2023-10-30T12:57:45.875242+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:45.875242+0000","last_undegraded":"2023-10-30T12:57:45.875242+0000","last_fullsized":"2023-10-30T12:57:45.875242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'347","last_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_deep_scrub":"53'347","last_deep_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.875104+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:05.779308+0000","scrub_duration":0.036461027999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37674,"num_omap_keys":69,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'540","reported_seq":2677,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:23.374500+0000","last_change":"2023-10-30T12:58:23.374581+0000","last_active":"2023-10-30T12:58:23.374500+0000","last_peered":"2023-10-30T12:58:23.374500+0000","last_clean":"2023-10-30T12:58:23.374500+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:58:23.374500+0000","last_undegraded":"2023-10-30T12:58:23.374500+0000","last_fullsized":"2023-10-30T12:58:23.374500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'540","last_scrub_stamp":"2023-10-30T12:58:23.374410+0000","last_deep_scrub":"53'540","last_deep_scrub_stamp":"2023-10-30T12:58:23.374410+0000","last_clean_scrub_stamp":"2023-10-30T12:58:23.374410+0000","objects_scrubbed":184,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:22:34.309284+0000","scrub_duration":0.167791421,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'35","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.423647+0000","last_change":"2023-10-30T12:57:58.423647+0000","last_active":"2023-10-30T12:57:58.423647+0000","last_peered":"2023-10-30T12:57:58.423647+0000","last_clean":"2023-10-30T12:57:58.423647+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:57:58.423647+0000","last_undegraded":"2023-10-30T12:57:58.423647+0000","last_fullsized":"2023-10-30T12:57:58.423647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.423320+0000","objects_scrubbed":3,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:35.335688+0000","scrub_duration":0.0090485700000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":33,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9500,"num_omap_keys":19,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'367","reported_seq":2492,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.672397+0000","last_change":"2023-10-30T12:52:38.194929+0000","last_active":"2023-10-30T12:57:39.672397+0000","last_peered":"2023-10-30T12:57:39.672397+0000","last_clean":"2023-10-30T12:57:39.672397+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:57:39.672397+0000","last_undegraded":"2023-10-30T12:57:39.672397+0000","last_fullsized":"2023-10-30T12:57:39.672397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'348","reported_seq":411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.293164+0000","last_change":"2023-10-30T12:57:57.930204+0000","last_active":"2023-10-30T12:58:15.293164+0000","last_peered":"2023-10-30T12:58:15.293164+0000","last_clean":"2023-10-30T12:58:15.293164+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:58:15.293164+0000","last_undegraded":"2023-10-30T12:58:15.293164+0000","last_fullsized":"2023-10-30T12:58:15.293164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'335","last_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_deep_scrub":"53'335","last_deep_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.929845+0000","objects_scrubbed":4,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:04.674566+0000","scrub_duration":0.019613695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2394351,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":1051,"num_write":344,"num_write_kb":2515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26754,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'519","reported_seq":2662,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.566278+0000","last_change":"2023-10-30T12:58:00.566341+0000","last_active":"2023-10-30T12:58:00.566278+0000","last_peered":"2023-10-30T12:58:00.566278+0000","last_clean":"2023-10-30T12:58:00.566278+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:58:00.566278+0000","last_undegraded":"2023-10-30T12:58:00.566278+0000","last_fullsized":"2023-10-30T12:58:00.566278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'519","last_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_deep_scrub":"53'519","last_deep_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.566191+0000","objects_scrubbed":170,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:02:51.046360+0000","scrub_duration":0.153203224,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"53'49","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.101254+0000","last_change":"2023-10-30T12:57:52.101317+0000","last_active":"2023-10-30T12:57:52.101254+0000","last_peered":"2023-10-30T12:57:52.101254+0000","last_clean":"2023-10-30T12:57:52.101254+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:52.101254+0000","last_undegraded":"2023-10-30T12:57:52.101254+0000","last_fullsized":"2023-10-30T12:57:52.101254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'49","last_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_deep_scrub":"53'49","last_deep_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.101149+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:45.571818+0000","scrub_duration":0.011136024,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26812,"num_omap_keys":51,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'498","reported_seq":2627,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.764619+0000","last_change":"2023-10-30T12:57:53.764679+0000","last_active":"2023-10-30T12:57:53.764619+0000","last_peered":"2023-10-30T12:57:53.764619+0000","last_clean":"2023-10-30T12:57:53.764619+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:53.764619+0000","last_undegraded":"2023-10-30T12:57:53.764619+0000","last_fullsized":"2023-10-30T12:57:53.764619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.764533+0000","objects_scrubbed":168,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:27.949277+0000","scrub_duration":0.31117073299999998,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'114","reported_seq":180,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.479616+0000","last_change":"2023-10-30T12:57:48.479616+0000","last_active":"2023-10-30T12:57:48.479616+0000","last_peered":"2023-10-30T12:57:48.479616+0000","last_clean":"2023-10-30T12:57:48.479616+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:57:48.479616+0000","last_undegraded":"2023-10-30T12:57:48.479616+0000","last_fullsized":"2023-10-30T12:57:48.479616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.479247+0000","objects_scrubbed":6,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:36.989062+0000","scrub_duration":0.019741666000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":108,"num_write_kb":4174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8868,"num_omap_keys":17,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'472","reported_seq":2592,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.006059+0000","last_change":"2023-10-30T12:58:13.006188+0000","last_active":"2023-10-30T12:58:13.006059+0000","last_peered":"2023-10-30T12:58:13.006059+0000","last_clean":"2023-10-30T12:58:13.006059+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:58:13.006059+0000","last_undegraded":"2023-10-30T12:58:13.006059+0000","last_fullsized":"2023-10-30T12:58:13.006059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'472","last_scrub_stamp":"2023-10-30T12:58:13.005836+0000","last_deep_scrub":"53'472","last_deep_scrub_stamp":"2023-10-30T12:58:13.005836+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.005836+0000","objects_scrubbed":155,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:43.557965+0000","scrub_duration":0.27633308600000001,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"53'363","reported_seq":434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.626957+0000","last_change":"2023-10-30T12:57:44.627005+0000","last_active":"2023-10-30T12:57:44.626957+0000","last_peered":"2023-10-30T12:57:44.626957+0000","last_clean":"2023-10-30T12:57:44.626957+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:44.626957+0000","last_undegraded":"2023-10-30T12:57:44.626957+0000","last_fullsized":"2023-10-30T12:57:44.626957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'363","last_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_deep_scrub":"53'363","last_deep_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.626874+0000","objects_scrubbed":9,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.857541+0000","scrub_duration":0.036431423999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31162,"num_omap_keys":58,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'521","reported_seq":2663,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.826274+0000","last_change":"2023-10-30T12:57:45.826319+0000","last_active":"2023-10-30T12:57:45.826274+0000","last_peered":"2023-10-30T12:57:45.826274+0000","last_clean":"2023-10-30T12:57:45.826274+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:45.826274+0000","last_undegraded":"2023-10-30T12:57:45.826274+0000","last_fullsized":"2023-10-30T12:57:45.826274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'521","last_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_deep_scrub":"53'521","last_deep_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.826203+0000","objects_scrubbed":179,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:48:53.989588+0000","scrub_duration":0.225502547,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"53'108","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.907067+0000","last_change":"2023-10-30T12:57:56.907176+0000","last_active":"2023-10-30T12:57:56.907067+0000","last_peered":"2023-10-30T12:57:56.907067+0000","last_clean":"2023-10-30T12:57:56.907067+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:56.907067+0000","last_undegraded":"2023-10-30T12:57:56.907067+0000","last_fullsized":"2023-10-30T12:57:56.907067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'108","last_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_deep_scrub":"53'108","last_deep_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.906852+0000","objects_scrubbed":4,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:54.254936+0000","scrub_duration":0.023033504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":7391,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19720,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'357","reported_seq":2495,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:39.668696+0000","last_change":"2023-10-30T12:52:38.168931+0000","last_active":"2023-10-30T12:57:39.668696+0000","last_peered":"2023-10-30T12:57:39.668696+0000","last_clean":"2023-10-30T12:57:39.668696+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:57:39.668696+0000","last_undegraded":"2023-10-30T12:57:39.668696+0000","last_fullsized":"2023-10-30T12:57:39.668696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'404","reported_seq":472,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.447273+0000","last_change":"2023-10-30T12:58:00.447405+0000","last_active":"2023-10-30T12:58:00.447273+0000","last_peered":"2023-10-30T12:58:00.447273+0000","last_clean":"2023-10-30T12:58:00.447273+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:58:00.447273+0000","last_undegraded":"2023-10-30T12:58:00.447273+0000","last_fullsized":"2023-10-30T12:58:00.447273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'404","last_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_deep_scrub":"53'404","last_deep_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.447060+0000","objects_scrubbed":6,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:18.599324+0000","scrub_duration":0.056206357999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":392,"num_write_kb":16593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'496","reported_seq":2628,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.739754+0000","last_change":"2023-10-30T12:58:07.739899+0000","last_active":"2023-10-30T12:58:07.739754+0000","last_peered":"2023-10-30T12:58:07.739754+0000","last_clean":"2023-10-30T12:58:07.739754+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:58:07.739754+0000","last_undegraded":"2023-10-30T12:58:07.739754+0000","last_fullsized":"2023-10-30T12:58:07.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:07.739536+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:07.739536+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.739536+0000","objects_scrubbed":167,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:03.366820+0000","scrub_duration":0.33077558200000001,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"53'125","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.181850+0000","last_change":"2023-10-30T12:57:44.181906+0000","last_active":"2023-10-30T12:57:44.181850+0000","last_peered":"2023-10-30T12:57:44.181850+0000","last_clean":"2023-10-30T12:57:44.181850+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:44.181850+0000","last_undegraded":"2023-10-30T12:57:44.181850+0000","last_fullsized":"2023-10-30T12:57:44.181850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.181771+0000","objects_scrubbed":13,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:04:46.051796+0000","scrub_duration":0.017190094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51810,"num_omap_keys":95,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'531","reported_seq":2665,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.175266+0000","last_change":"2023-10-30T12:58:15.175320+0000","last_active":"2023-10-30T12:58:15.175266+0000","last_peered":"2023-10-30T12:58:15.175266+0000","last_clean":"2023-10-30T12:58:15.175266+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:58:15.175266+0000","last_undegraded":"2023-10-30T12:58:15.175266+0000","last_fullsized":"2023-10-30T12:58:15.175266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'531","last_scrub_stamp":"2023-10-30T12:58:15.175195+0000","last_deep_scrub":"53'531","last_deep_scrub_stamp":"2023-10-30T12:58:15.175195+0000","last_clean_scrub_stamp":"2023-10-30T12:58:15.175195+0000","objects_scrubbed":174,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:47.138672+0000","scrub_duration":0.194958246,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'107","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.532064+0000","last_change":"2023-10-30T12:57:48.532178+0000","last_active":"2023-10-30T12:57:48.532064+0000","last_peered":"2023-10-30T12:57:48.532064+0000","last_clean":"2023-10-30T12:57:48.532064+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:48.532064+0000","last_undegraded":"2023-10-30T12:57:48.532064+0000","last_fullsized":"2023-10-30T12:57:48.532064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'107","last_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_deep_scrub":"53'107","last_deep_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.531861+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:05.413034+0000","scrub_duration":0.032263541999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":179,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'467","reported_seq":2603,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.047339+0000","last_change":"2023-10-30T12:58:00.047393+0000","last_active":"2023-10-30T12:58:00.047339+0000","last_peered":"2023-10-30T12:58:00.047339+0000","last_clean":"2023-10-30T12:58:00.047339+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:58:00.047339+0000","last_undegraded":"2023-10-30T12:58:00.047339+0000","last_fullsized":"2023-10-30T12:58:00.047339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'467","last_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_deep_scrub":"53'467","last_deep_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.047244+0000","objects_scrubbed":156,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:00.326328+0000","scrub_duration":0.23210165099999999,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12072627,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":307,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'170","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:59.087103+0000","last_change":"2023-10-30T12:57:59.087197+0000","last_active":"2023-10-30T12:57:59.087103+0000","last_peered":"2023-10-30T12:57:59.087103+0000","last_clean":"2023-10-30T12:57:59.087103+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:57:59.087103+0000","last_undegraded":"2023-10-30T12:57:59.087103+0000","last_fullsized":"2023-10-30T12:57:59.087103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'170","last_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_deep_scrub":"53'170","last_deep_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_clean_scrub_stamp":"2023-10-30T12:57:59.087009+0000","objects_scrubbed":5,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:31.789859+0000","scrub_duration":0.011509221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32668,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'502","reported_seq":2637,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.084264+0000","last_change":"2023-10-30T12:58:09.084320+0000","last_active":"2023-10-30T12:58:09.084264+0000","last_peered":"2023-10-30T12:58:09.084264+0000","last_clean":"2023-10-30T12:58:09.084264+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:58:09.084264+0000","last_undegraded":"2023-10-30T12:58:09.084264+0000","last_fullsized":"2023-10-30T12:58:09.084264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'502","last_scrub_stamp":"2023-10-30T12:58:09.084184+0000","last_deep_scrub":"53'502","last_deep_scrub_stamp":"2023-10-30T12:58:09.084184+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.084184+0000","objects_scrubbed":163,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:23:56.040279+0000","scrub_duration":0.29045587899999997,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"53'127","reported_seq":187,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:08.170192+0000","last_change":"2023-10-30T12:58:08.170243+0000","last_active":"2023-10-30T12:58:08.170192+0000","last_peered":"2023-10-30T12:58:08.170192+0000","last_clean":"2023-10-30T12:58:08.170192+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:58:08.170192+0000","last_undegraded":"2023-10-30T12:58:08.170192+0000","last_fullsized":"2023-10-30T12:58:08.170192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'127","last_scrub_stamp":"2023-10-30T12:58:08.170113+0000","last_deep_scrub":"53'127","last_deep_scrub_stamp":"2023-10-30T12:58:08.170113+0000","last_clean_scrub_stamp":"2023-10-30T12:58:08.170113+0000","objects_scrubbed":4,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:16:44.348618+0000","scrub_duration":0.016712257000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20202,"num_omap_keys":37,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'561","reported_seq":2697,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.335935+0000","last_change":"2023-10-30T12:58:13.336055+0000","last_active":"2023-10-30T12:58:13.335935+0000","last_peered":"2023-10-30T12:58:13.335935+0000","last_clean":"2023-10-30T12:58:13.335935+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:58:13.335935+0000","last_undegraded":"2023-10-30T12:58:13.335935+0000","last_fullsized":"2023-10-30T12:58:13.335935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'561","last_scrub_stamp":"2023-10-30T12:58:13.335723+0000","last_deep_scrub":"53'561","last_deep_scrub_stamp":"2023-10-30T12:58:13.335723+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.335723+0000","objects_scrubbed":194,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:26.649100+0000","scrub_duration":0.32796992200000002,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526751,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":358,"num_write_kb":14373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"53'19","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.062670+0000","last_change":"2023-10-30T12:58:09.062763+0000","last_active":"2023-10-30T12:58:09.062670+0000","last_peered":"2023-10-30T12:58:09.062670+0000","last_clean":"2023-10-30T12:58:09.062670+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:58:09.062670+0000","last_undegraded":"2023-10-30T12:58:09.062670+0000","last_fullsized":"2023-10-30T12:58:09.062670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2023-10-30T12:58:09.062573+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2023-10-30T12:58:09.062573+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.062573+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:11:55.643732+0000","scrub_duration":0.0070198639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30824,"num_omap_keys":57,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'496","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:17.968850+0000","last_change":"2023-10-30T12:58:17.968907+0000","last_active":"2023-10-30T12:58:17.968850+0000","last_peered":"2023-10-30T12:58:17.968850+0000","last_clean":"2023-10-30T12:58:17.968850+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:58:17.968850+0000","last_undegraded":"2023-10-30T12:58:17.968850+0000","last_fullsized":"2023-10-30T12:58:17.968850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:17.968753+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:17.968753+0000","last_clean_scrub_stamp":"2023-10-30T12:58:17.968753+0000","objects_scrubbed":163,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:15:36.268957+0000","scrub_duration":0.23173764799999999,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468449,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":323,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.811840+0000","last_change":"2023-10-30T12:57:48.811953+0000","last_active":"2023-10-30T12:57:48.811840+0000","last_peered":"2023-10-30T12:57:48.811840+0000","last_clean":"2023-10-30T12:57:48.811840+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:48.811840+0000","last_undegraded":"2023-10-30T12:57:48.811840+0000","last_fullsized":"2023-10-30T12:57:48.811840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.811641+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:27.857700+0000","scrub_duration":0.0086407700000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'15","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.871134+0000","last_change":"2023-10-30T12:57:49.871261+0000","last_active":"2023-10-30T12:57:49.871134+0000","last_peered":"2023-10-30T12:57:49.871134+0000","last_clean":"2023-10-30T12:57:49.871134+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:49.871134+0000","last_undegraded":"2023-10-30T12:57:49.871134+0000","last_fullsized":"2023-10-30T12:57:49.871134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.870906+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:10:34.313354+0000","scrub_duration":0.010713916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15500,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'456","reported_seq":2587,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:21.594862+0000","last_change":"2023-10-30T12:58:21.594938+0000","last_active":"2023-10-30T12:58:21.594862+0000","last_peered":"2023-10-30T12:58:21.594862+0000","last_clean":"2023-10-30T12:58:21.594862+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:58:21.594862+0000","last_undegraded":"2023-10-30T12:58:21.594862+0000","last_fullsized":"2023-10-30T12:58:21.594862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'456","last_scrub_stamp":"2023-10-30T12:58:21.594787+0000","last_deep_scrub":"53'456","last_deep_scrub_stamp":"2023-10-30T12:58:21.594787+0000","last_clean_scrub_stamp":"2023-10-30T12:58:21.594787+0000","objects_scrubbed":146,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:49.424663+0000","scrub_duration":0.12018036999999999,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"53'505","reported_seq":574,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.786677+0000","last_change":"2023-10-30T12:57:50.786811+0000","last_active":"2023-10-30T12:57:50.786677+0000","last_peered":"2023-10-30T12:57:50.786677+0000","last_clean":"2023-10-30T12:57:50.786677+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:50.786677+0000","last_undegraded":"2023-10-30T12:57:50.786677+0000","last_fullsized":"2023-10-30T12:57:50.786677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.786437+0000","objects_scrubbed":8,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:51.266608+0000","scrub_duration":0.043680202000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":44444,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'489","reported_seq":2612,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:06.635929+0000","last_change":"2023-10-30T12:58:06.635989+0000","last_active":"2023-10-30T12:58:06.635929+0000","last_peered":"2023-10-30T12:58:06.635929+0000","last_clean":"2023-10-30T12:58:06.635929+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:58:06.635929+0000","last_undegraded":"2023-10-30T12:58:06.635929+0000","last_fullsized":"2023-10-30T12:58:06.635929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'489","last_scrub_stamp":"2023-10-30T12:58:06.635853+0000","last_deep_scrub":"53'489","last_deep_scrub_stamp":"2023-10-30T12:58:06.635853+0000","last_clean_scrub_stamp":"2023-10-30T12:58:06.635853+0000","objects_scrubbed":162,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:55:34.992256+0000","scrub_duration":0.25796478499999997,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'161","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.887087+0000","last_change":"2023-10-30T12:57:47.115495+0000","last_active":"2023-10-30T12:58:19.887087+0000","last_peered":"2023-10-30T12:58:19.887087+0000","last_clean":"2023-10-30T12:58:19.887087+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:58:19.887087+0000","last_undegraded":"2023-10-30T12:58:19.887087+0000","last_fullsized":"2023-10-30T12:58:19.887087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_clean_scrub_stamp":"2023-10-30T12:57:47.115321+0000","objects_scrubbed":9,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:48:57.832139+0000","scrub_duration":0.015406534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4552496,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":150,"num_write_kb":4602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63882,"num_omap_keys":117,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'493","reported_seq":2635,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:11.291688+0000","last_change":"2023-10-30T12:58:11.291745+0000","last_active":"2023-10-30T12:58:11.291688+0000","last_peered":"2023-10-30T12:58:11.291688+0000","last_clean":"2023-10-30T12:58:11.291688+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:58:11.291688+0000","last_undegraded":"2023-10-30T12:58:11.291688+0000","last_fullsized":"2023-10-30T12:58:11.291688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-30T12:58:11.291608+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-30T12:58:11.291608+0000","last_clean_scrub_stamp":"2023-10-30T12:58:11.291608+0000","objects_scrubbed":168,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:34:58.191035+0000","scrub_duration":0.14191978399999999,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"53'249","reported_seq":311,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.994818+0000","last_change":"2023-10-30T12:58:00.994936+0000","last_active":"2023-10-30T12:58:00.994818+0000","last_peered":"2023-10-30T12:58:00.994818+0000","last_clean":"2023-10-30T12:58:00.994818+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:58:00.994818+0000","last_undegraded":"2023-10-30T12:58:00.994818+0000","last_fullsized":"2023-10-30T12:58:00.994818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'249","last_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_deep_scrub":"53'249","last_deep_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.994611+0000","objects_scrubbed":4,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:45.189479+0000","scrub_duration":0.037553993000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21832,"num_omap_keys":40,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'541","reported_seq":2671,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.766867+0000","last_change":"2023-10-30T12:57:52.767026+0000","last_active":"2023-10-30T12:57:52.766867+0000","last_peered":"2023-10-30T12:57:52.766867+0000","last_clean":"2023-10-30T12:57:52.766867+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:52.766867+0000","last_undegraded":"2023-10-30T12:57:52.766867+0000","last_fullsized":"2023-10-30T12:57:52.766867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'541","last_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_deep_scrub":"53'541","last_deep_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.766665+0000","objects_scrubbed":172,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:47:06.068606+0000","scrub_duration":0.32093353800000002,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'4","reported_seq":64,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.124470+0000","last_change":"2023-10-30T12:58:09.124515+0000","last_active":"2023-10-30T12:58:09.124470+0000","last_peered":"2023-10-30T12:58:09.124470+0000","last_clean":"2023-10-30T12:58:09.124470+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:58:09.124470+0000","last_undegraded":"2023-10-30T12:58:09.124470+0000","last_fullsized":"2023-10-30T12:58:09.124470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2023-10-30T12:58:09.124398+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2023-10-30T12:58:09.124398+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.124398+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:50:41.929762+0000","scrub_duration":0.003007633,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13058,"num_omap_keys":24,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'330","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797314+0000","last_change":"2023-10-30T12:52:38.195417+0000","last_active":"2023-10-30T12:57:45.797314+0000","last_peered":"2023-10-30T12:57:45.797314+0000","last_clean":"2023-10-30T12:57:45.797314+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:57:45.797314+0000","last_undegraded":"2023-10-30T12:57:45.797314+0000","last_fullsized":"2023-10-30T12:57:45.797314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9698880,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":313,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'15","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.813043+0000","last_change":"2023-10-30T12:57:58.813159+0000","last_active":"2023-10-30T12:57:58.813043+0000","last_peered":"2023-10-30T12:57:58.813043+0000","last_clean":"2023-10-30T12:57:58.813043+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:58.813043+0000","last_undegraded":"2023-10-30T12:57:58.813043+0000","last_fullsized":"2023-10-30T12:57:58.813043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.812837+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:16.476751+0000","scrub_duration":0.009639435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25372,"num_omap_keys":48,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'514","reported_seq":2650,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.292021+0000","last_change":"2023-10-30T12:58:22.292097+0000","last_active":"2023-10-30T12:58:22.292021+0000","last_peered":"2023-10-30T12:58:22.292021+0000","last_clean":"2023-10-30T12:58:22.292021+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:58:22.292021+0000","last_undegraded":"2023-10-30T12:58:22.292021+0000","last_fullsized":"2023-10-30T12:58:22.292021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'514","last_scrub_stamp":"2023-10-30T12:58:22.291946+0000","last_deep_scrub":"53'514","last_deep_scrub_stamp":"2023-10-30T12:58:22.291946+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.291946+0000","objects_scrubbed":173,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:03.363935+0000","scrub_duration":0.133602522,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'102","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.157983+0000","last_change":"2023-10-30T12:58:10.157983+0000","last_active":"2023-10-30T12:58:10.157983+0000","last_peered":"2023-10-30T12:58:10.157983+0000","last_clean":"2023-10-30T12:58:10.157983+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:58:10.157983+0000","last_undegraded":"2023-10-30T12:58:10.157983+0000","last_fullsized":"2023-10-30T12:58:10.157983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2023-10-30T12:58:10.157787+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2023-10-30T12:58:10.157787+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.157787+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:54:54.361702+0000","scrub_duration":0.053518498999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":240,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11565,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'451","reported_seq":2580,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:08.583430+0000","last_change":"2023-10-30T12:58:08.583430+0000","last_active":"2023-10-30T12:58:08.583430+0000","last_peered":"2023-10-30T12:58:08.583430+0000","last_clean":"2023-10-30T12:58:08.583430+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:58:08.583430+0000","last_undegraded":"2023-10-30T12:58:08.583430+0000","last_fullsized":"2023-10-30T12:58:08.583430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'451","last_scrub_stamp":"2023-10-30T12:58:08.583306+0000","last_deep_scrub":"53'451","last_deep_scrub_stamp":"2023-10-30T12:58:08.583306+0000","last_clean_scrub_stamp":"2023-10-30T12:58:08.583306+0000","objects_scrubbed":138,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:46:20.024190+0000","scrub_duration":0.12646469299999999,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"53'16","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:51.452812+0000","last_change":"2023-10-30T12:57:51.452812+0000","last_active":"2023-10-30T12:57:51.452812+0000","last_peered":"2023-10-30T12:57:51.452812+0000","last_clean":"2023-10-30T12:57:51.452812+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:51.452812+0000","last_undegraded":"2023-10-30T12:57:51.452812+0000","last_fullsized":"2023-10-30T12:57:51.452812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_clean_scrub_stamp":"2023-10-30T12:57:51.452333+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:18:15.321071+0000","scrub_duration":0.011134068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4360,"num_omap_keys":8,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'484","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.782635+0000","last_change":"2023-10-30T12:57:48.782635+0000","last_active":"2023-10-30T12:57:48.782635+0000","last_peered":"2023-10-30T12:57:48.782635+0000","last_clean":"2023-10-30T12:57:48.782635+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:48.782635+0000","last_undegraded":"2023-10-30T12:57:48.782635+0000","last_fullsized":"2023-10-30T12:57:48.782635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.782504+0000","objects_scrubbed":167,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:48.560996+0000","scrub_duration":0.129876255,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'167","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.808170+0000","last_change":"2023-10-30T12:57:49.808170+0000","last_active":"2023-10-30T12:57:49.808170+0000","last_peered":"2023-10-30T12:57:49.808170+0000","last_clean":"2023-10-30T12:57:49.808170+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:49.808170+0000","last_undegraded":"2023-10-30T12:57:49.808170+0000","last_fullsized":"2023-10-30T12:57:49.808170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'167","last_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_deep_scrub":"53'167","last_deep_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.807844+0000","objects_scrubbed":4,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:19.047233+0000","scrub_duration":0.025125933999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6708,"num_omap_keys":13,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'495","reported_seq":2620,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:54.741397+0000","last_change":"2023-10-30T12:57:54.741397+0000","last_active":"2023-10-30T12:57:54.741397+0000","last_peered":"2023-10-30T12:57:54.741397+0000","last_clean":"2023-10-30T12:57:54.741397+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:54.741397+0000","last_undegraded":"2023-10-30T12:57:54.741397+0000","last_fullsized":"2023-10-30T12:57:54.741397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_clean_scrub_stamp":"2023-10-30T12:57:54.741074+0000","objects_scrubbed":156,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:22.048659+0000","scrub_duration":0.30326803000000002,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"53'67","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:06.182656+0000","last_change":"2023-10-30T12:58:06.182656+0000","last_active":"2023-10-30T12:58:06.182656+0000","last_peered":"2023-10-30T12:58:06.182656+0000","last_clean":"2023-10-30T12:58:06.182656+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:58:06.182656+0000","last_undegraded":"2023-10-30T12:58:06.182656+0000","last_fullsized":"2023-10-30T12:58:06.182656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'67","last_scrub_stamp":"2023-10-30T12:58:06.182512+0000","last_deep_scrub":"53'67","last_deep_scrub_stamp":"2023-10-30T12:58:06.182512+0000","last_clean_scrub_stamp":"2023-10-30T12:58:06.182512+0000","objects_scrubbed":9,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:52:30.629159+0000","scrub_duration":0.023539636999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":61026,"num_omap_keys":114,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'487","reported_seq":2625,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.178881+0000","last_change":"2023-10-30T12:58:13.178938+0000","last_active":"2023-10-30T12:58:13.178881+0000","last_peered":"2023-10-30T12:58:13.178881+0000","last_clean":"2023-10-30T12:58:13.178881+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:58:13.178881+0000","last_undegraded":"2023-10-30T12:58:13.178881+0000","last_fullsized":"2023-10-30T12:58:13.178881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:13.178803+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:13.178803+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.178803+0000","objects_scrubbed":160,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:04:19.440362+0000","scrub_duration":0.131130357,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'122","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.127400+0000","last_change":"2023-10-30T12:58:00.127400+0000","last_active":"2023-10-30T12:58:00.127400+0000","last_peered":"2023-10-30T12:58:00.127400+0000","last_clean":"2023-10-30T12:58:00.127400+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:58:00.127400+0000","last_undegraded":"2023-10-30T12:58:00.127400+0000","last_fullsized":"2023-10-30T12:58:00.127400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'122","last_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_deep_scrub":"53'122","last_deep_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.127231+0000","objects_scrubbed":4,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:56.254642+0000","scrub_duration":0.026565129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":199,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18104,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'486","reported_seq":2614,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.540348+0000","last_change":"2023-10-30T12:58:19.540393+0000","last_active":"2023-10-30T12:58:19.540348+0000","last_peered":"2023-10-30T12:58:19.540348+0000","last_clean":"2023-10-30T12:58:19.540348+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:58:19.540348+0000","last_undegraded":"2023-10-30T12:58:19.540348+0000","last_fullsized":"2023-10-30T12:58:19.540348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'486","last_scrub_stamp":"2023-10-30T12:58:19.540267+0000","last_deep_scrub":"53'486","last_deep_scrub_stamp":"2023-10-30T12:58:19.540267+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.540267+0000","objects_scrubbed":156,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:44:39.118000+0000","scrub_duration":0.122920026,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"53'192","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.821533+0000","last_change":"2023-10-30T12:57:56.821734+0000","last_active":"2023-10-30T12:57:56.821533+0000","last_peered":"2023-10-30T12:57:56.821533+0000","last_clean":"2023-10-30T12:57:56.821533+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:57:56.821533+0000","last_undegraded":"2023-10-30T12:57:56.821533+0000","last_fullsized":"2023-10-30T12:57:56.821533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'192","last_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_deep_scrub":"53'192","last_deep_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.821278+0000","objects_scrubbed":5,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:54:26.335029+0000","scrub_duration":0.022744626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":4107,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'488","reported_seq":2623,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.602376+0000","last_change":"2023-10-30T12:58:01.602376+0000","last_active":"2023-10-30T12:58:01.602376+0000","last_peered":"2023-10-30T12:58:01.602376+0000","last_clean":"2023-10-30T12:58:01.602376+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:58:01.602376+0000","last_undegraded":"2023-10-30T12:58:01.602376+0000","last_fullsized":"2023-10-30T12:58:01.602376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'488","last_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_deep_scrub":"53'488","last_deep_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.602238+0000","objects_scrubbed":157,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:01.459702+0000","scrub_duration":0.22063908099999999,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'334","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.797779+0000","last_change":"2023-10-30T12:52:38.195758+0000","last_active":"2023-10-30T12:57:45.797779+0000","last_peered":"2023-10-30T12:57:45.797779+0000","last_clean":"2023-10-30T12:57:45.797779+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:57:45.797779+0000","last_undegraded":"2023-10-30T12:57:45.797779+0000","last_fullsized":"2023-10-30T12:57:45.797779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T12:45:18.050351+0000","last_clean_scrub_stamp":"2023-10-30T12:45:18.050351+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22516204,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":313,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"53'154","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.745224+0000","last_change":"2023-10-30T12:57:53.745301+0000","last_active":"2023-10-30T12:57:53.745224+0000","last_peered":"2023-10-30T12:57:53.745224+0000","last_clean":"2023-10-30T12:57:53.745224+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:57:53.745224+0000","last_undegraded":"2023-10-30T12:57:53.745224+0000","last_fullsized":"2023-10-30T12:57:53.745224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.745078+0000","objects_scrubbed":3,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:28:27.394467+0000","scrub_duration":0.016168252000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'487","reported_seq":2625,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.211256+0000","last_change":"2023-10-30T12:58:15.211299+0000","last_active":"2023-10-30T12:58:15.211256+0000","last_peered":"2023-10-30T12:58:15.211256+0000","last_clean":"2023-10-30T12:58:15.211256+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:58:15.211256+0000","last_undegraded":"2023-10-30T12:58:15.211256+0000","last_fullsized":"2023-10-30T12:58:15.211256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:15.211165+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:15.211165+0000","last_clean_scrub_stamp":"2023-10-30T12:58:15.211165+0000","objects_scrubbed":152,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:05:41.398922+0000","scrub_duration":0.18129031300000001,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242846,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":314,"num_write_kb":14255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"53'369","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.867694+0000","last_change":"2023-10-30T12:57:55.867838+0000","last_active":"2023-10-30T12:57:55.867694+0000","last_peered":"2023-10-30T12:57:55.867694+0000","last_clean":"2023-10-30T12:57:55.867694+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:57:55.867694+0000","last_undegraded":"2023-10-30T12:57:55.867694+0000","last_fullsized":"2023-10-30T12:57:55.867694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'369","last_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_deep_scrub":"53'369","last_deep_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.867484+0000","objects_scrubbed":7,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:44.299714+0000","scrub_duration":0.065173216000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":22,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'482","reported_seq":2610,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:14.587594+0000","last_change":"2023-10-30T12:58:14.587674+0000","last_active":"2023-10-30T12:58:14.587594+0000","last_peered":"2023-10-30T12:58:14.587594+0000","last_clean":"2023-10-30T12:58:14.587594+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:58:14.587594+0000","last_undegraded":"2023-10-30T12:58:14.587594+0000","last_fullsized":"2023-10-30T12:58:14.587594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'482","last_scrub_stamp":"2023-10-30T12:58:14.587519+0000","last_deep_scrub":"53'482","last_deep_scrub_stamp":"2023-10-30T12:58:14.587519+0000","last_clean_scrub_stamp":"2023-10-30T12:58:14.587519+0000","objects_scrubbed":147,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:39:51.524443+0000","scrub_duration":0.129591384,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"53'281","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.503383+0000","last_change":"2023-10-30T12:57:53.503441+0000","last_active":"2023-10-30T12:57:53.503383+0000","last_peered":"2023-10-30T12:57:53.503383+0000","last_clean":"2023-10-30T12:57:53.503383+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:53.503383+0000","last_undegraded":"2023-10-30T12:57:53.503383+0000","last_fullsized":"2023-10-30T12:57:53.503383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'281","last_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_deep_scrub":"53'281","last_deep_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.503283+0000","objects_scrubbed":3,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:22.524008+0000","scrub_duration":0.021426580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8098,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'489","reported_seq":2617,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.723690+0000","last_change":"2023-10-30T12:58:10.723733+0000","last_active":"2023-10-30T12:58:10.723690+0000","last_peered":"2023-10-30T12:58:10.723690+0000","last_clean":"2023-10-30T12:58:10.723690+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:58:10.723690+0000","last_undegraded":"2023-10-30T12:58:10.723690+0000","last_fullsized":"2023-10-30T12:58:10.723690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'489","last_scrub_stamp":"2023-10-30T12:58:10.723608+0000","last_deep_scrub":"53'489","last_deep_scrub_stamp":"2023-10-30T12:58:10.723608+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.723608+0000","objects_scrubbed":158,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:38:16.558220+0000","scrub_duration":0.19531570600000001,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"53'201","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.639282+0000","last_change":"2023-10-30T12:57:43.639282+0000","last_active":"2023-10-30T12:57:43.639282+0000","last_peered":"2023-10-30T12:57:43.639282+0000","last_clean":"2023-10-30T12:57:43.639282+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:57:43.639282+0000","last_undegraded":"2023-10-30T12:57:43.639282+0000","last_fullsized":"2023-10-30T12:57:43.639282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'201","last_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_deep_scrub":"53'201","last_deep_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_clean_scrub_stamp":"2023-10-30T12:57:43.639154+0000","objects_scrubbed":8,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:04.340919+0000","scrub_duration":0.033784741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925184388,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20887,"num_write_kb":1047691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902518272,"data_stored":1848828680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31022,"ondisk_log_size":31022,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":177746842,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":601,"num_read_kb":143265,"num_write":5698,"num_write_kb":179360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":692939,"num_omap_keys":1293,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":355655680,"data_stored":355493684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5593,"ondisk_log_size":5593,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398996,"kb_used_data":327564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93972844,"statfs":{"total":96636764160,"available":96228192256,"internally_reserved":0,"allocated":335425536,"data_stored":327567493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1140000000000001,"15min":0.89600000000000002},"min":{"1min":0.62,"5min":0.44,"15min":0.436},"max":{"1min":2.0499999999999998,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.794,"5min":1.3180000000000001,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":17.966999999999999,"5min":25.327000000000002,"15min":25.327000000000002},"last":1.0029999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.135,"15min":0.93700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.296,"15min":1.0489999999999999},"min":{"1min":0.73099999999999998,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.5720000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.2889999999999999,"15min":1.083},"min":{"1min":0.621,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":10.972,"5min":28.881,"15min":28.881},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.29,"15min":1.0589999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.784,"5min":25.364000000000001,"15min":25.364000000000001},"last":1.129}]},{"osd":3,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.236,"15min":0.98799999999999999},"min":{"1min":0.65900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":3.8660000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.615,"5min":1.2969999999999999,"15min":1.073},"min":{"1min":0.72699999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":14.754,"5min":28.922000000000001,"15min":28.922000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.319,"15min":1.135},"min":{"1min":0.77300000000000002,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":2.6200000000000001,"5min":25.253,"15min":25.253},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.363,"15min":1.2050000000000001},"min":{"1min":0.85199999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":3.7709999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4219999999999999,"15min":1.214},"min":{"1min":0.69999999999999996,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":8.6349999999999998,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.099},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.2310000000000001,"15min":1.1870000000000001},"min":{"1min":0.77300000000000002,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.1719999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.4450000000000001,"15min":1.2969999999999999},"min":{"1min":0.872,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.5429999999999999,"5min":26.398,"15min":26.398},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.496,"15min":1.3260000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":1.8029999999999999,"5min":25.401,"15min":25.401},"last":1.302}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":291784,"kb_used_data":238400,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94080056,"statfs":{"total":96636764160,"available":96337977344,"internally_reserved":0,"allocated":244121600,"data_stored":238900978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.504},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83299999999999996,"15min":0.85199999999999998},"min":{"1min":0.55700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":0.93000000000000005,"15min":0.83799999999999997},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":11.834,"5min":11.834,"15min":11.834},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.65400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3140000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":1.028}]},{"osd":2,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.46100000000000002},"max":{"1min":1.4470000000000001,"5min":2.028,"15min":2.028},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":0.98399999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.4259999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.97099999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0820000000000001,"15min":1.026},"min":{"1min":0.85699999999999998,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.044,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.76600000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.97599999999999998,"15min":1.0249999999999999},"min":{"1min":0.89700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.2170000000000001,"15min":1.179},"min":{"1min":0.69999999999999996,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.724}]},{"osd":5,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":2.0009999999999999,"5min":1.2030000000000001,"15min":1.0900000000000001},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":20.446000000000002,"5min":20.446000000000002,"15min":20.446000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.135,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6369999999999996,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.3720000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.73399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.595,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.665},{"interface":"front","average":{"1min":3.9359999999999999,"5min":1.8500000000000001,"15min":1.454},"min":{"1min":1.0329999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":50.406999999999996,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.8360000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373548,"kb_used_data":293796,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93998292,"statfs":{"total":96636764160,"available":96254251008,"internally_reserved":0,"allocated":300847104,"data_stored":293077245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84999999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.149,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.89300000000000002,"15min":0.93100000000000005},"min":{"1min":0.52700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.385,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":0.69399999999999995}]},{"osd":1,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":3.258},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2549999999999999,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.83699999999999997}]},{"osd":2,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2549999999999999,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.042,"15min":1.014},"min":{"1min":0.63600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.266,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":1.0960000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:57:28 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.02,"15min":1.0469999999999999},"min":{"1min":0.754,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3400000000000001,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.165},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.024,"15min":1.054},"min":{"1min":0.59499999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4370000000000001,"5min":2.5190000000000001,"15min":3.524},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0620000000000001},"min":{"1min":0.627,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.397,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1479999999999999},"min":{"1min":0.96499999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.2170000000000001}]},{"osd":6,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.137,"15min":1.139},"min":{"1min":0.755,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5800000000000001,"5min":2.802,"15min":2.802},"last":1.26},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.1379999999999999,"15min":1.123},"min":{"1min":0.67700000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5580000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:57:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.131,"15min":1.1919999999999999},"min":{"1min":0.76400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.512,"5min":2.3940000000000001,"15min":2.508},"last":1.133},{"interface":"front","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.2549999999999999},"min":{"1min":1.1240000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5329999999999999,"5min":2.137,"15min":2.137},"last":1.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409504,"kb_used_data":342232,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93962336,"statfs":{"total":96636764160,"available":96217432064,"internally_reserved":0,"allocated":350445568,"data_stored":342284545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90600000000000003,"15min":0.88500000000000001},"min":{"1min":0.60499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":2.819,"15min":2.819},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2809999999999999,"5min":2.859,"15min":2.859},"last":1.302}]},{"osd":1,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91900000000000004,"15min":0.89300000000000002},"min":{"1min":0.55000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3140000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92900000000000005,"15min":0.95199999999999996},"min":{"1min":0.56100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.1479999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.1160000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.96299999999999997},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1020000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.018,"15min":0.98499999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.55700000000000005},"max":{"1min":1.226,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.218},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0209999999999999,"15min":1.0409999999999999},"min":{"1min":0.67400000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.179,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0720000000000001,"15min":1.087},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2170000000000001,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0009999999999999,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1870000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2569999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.097,"15min":1.1459999999999999},"min":{"1min":0.73599999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.306,"5min":1.595,"15min":2.1360000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1830000000000001,"15min":1.202},"min":{"1min":0.86399999999999999,"5min":0.81200000000000006,"15min":0.65000000000000002},"max":{"1min":1.3979999999999999,"5min":1.661,"15min":2.093},"last":1.0169999999999999}]},{"osd":7,"last update":"Mon Oct 30 12:57:39 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.2070000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.73199999999999998},"max":{"1min":1.462,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.355,"5min":1.7769999999999999,"15min":1.851},"last":0.96299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575010,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":402220,"kb_used_data":334948,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93969620,"statfs":{"total":96636764160,"available":96224890880,"internally_reserved":0,"allocated":342986752,"data_stored":334679455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68876744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.61299999999999999,"15min":0.50800000000000001},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":11.262,"15min":11.262},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.65100000000000002,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":11.244,"15min":11.244},"last":0.499}]},{"osd":1,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55600000000000005,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.60199999999999998,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.438}]},{"osd":2,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.372,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Oct 30 12:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.499},"max":{"1min":0.98099999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83799999999999997},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.226,"5min":1.786,"15min":1.867},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.337,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.98999999999999999,"15min":0.92200000000000004},"min":{"1min":0.78100000000000003,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.3720000000000001,"5min":3.125,"15min":3.125},"last":0.90900000000000003}]},{"osd":6,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.88600000000000001},"min":{"1min":0.73899999999999999,"5min":0.63400000000000001,"15min":0.54700000000000004},"max":{"1min":1.2290000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.61199999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3260000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Oct 30 12:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2130000000000001,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.88600000000000001,"15min":0.89300000000000002},"min":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2709999999999999,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":204264,"kb_used_data":150880,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94167576,"statfs":{"total":96636764160,"available":96427597824,"internally_reserved":0,"allocated":154501120,"data_stored":149421723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.311},"max":{"1min":0.70199999999999996,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.41699999999999998}]},{"osd":1,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.84499999999999997,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53100000000000003,"15min":0.82099999999999995},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80800000000000005,"15min":0.876},"min":{"1min":0.66000000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80600000000000005,"15min":0.96699999999999997},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.165,"5min":1.165,"15min":34.307000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2290000000000001,"5min":1.8959999999999999,"15min":48.951000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83199999999999996,"15min":1.014},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1439999999999999,"5min":1.827,"15min":27.899000000000001},"last":0.754}]},{"osd":6,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.85499999999999998,"15min":1.0089999999999999},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.264,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":0.80700000000000005}]},{"osd":7,"last update":"Mon Oct 30 12:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.877,"15min":1.077},"min":{"1min":0.753,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2829999999999999,"5min":1.337,"15min":34.420000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":1.171},"min":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.59099999999999997},"max":{"1min":1.3160000000000001,"5min":1.974,"15min":40.710000000000001},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":270504,"kb_used_data":212960,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94101336,"statfs":{"total":96636764160,"available":96359768064,"internally_reserved":0,"allocated":218071040,"data_stored":212483944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.26500000000000001,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":0.58799999999999997,"5min":26.349,"15min":26.349},"last":0.316},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.73699999999999999,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.70499999999999996,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.67400000000000004,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57599999999999996,"15min":0.69899999999999995},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.32800000000000001},"max":{"1min":0.622,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91200000000000003,"15min":0.93000000000000005},"min":{"1min":0.623,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.0509999999999999,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.60199999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96599999999999997,"5min":15.653,"15min":15.653},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.92700000000000005,"15min":0.89400000000000002},"min":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0589999999999999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.97799999999999998,"15min":1.044},"min":{"1min":0.627,"5min":0.62,"15min":0.48099999999999998},"max":{"1min":1.056,"5min":12.15,"15min":12.15},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82099999999999995,"15min":0.96699999999999997},"min":{"1min":0.61199999999999999,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":1.04,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0009999999999999,"15min":1.0780000000000001},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":15.766999999999999,"15min":15.766999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 30 12:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.006,"15min":1.079},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.103,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86699999999999999,"15min":1.0169999999999999},"min":{"1min":0.63400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":0.82999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":377664,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93994176,"statfs":{"total":96636764160,"available":96250036224,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":1.732,"15min":1.732},"last":0.63100000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.33200000000000002,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.738,"15min":1.738},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.94799999999999995,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.96399999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.64200000000000002,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":1.851,"15min":1.851},"last":0.68200000000000005}]},{"osd":4,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.94599999999999995,"15min":0.873},"min":{"1min":0.60999999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":1.4419999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.84399999999999997},"min":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94499999999999995,"15min":0.90000000000000002},"min":{"1min":0.76800000000000002,"5min":0.58299999999999996,"15min":0.441},"max":{"1min":1.593,"5min":1.704,"15min":1.704},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66100000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.82699999999999996}]},{"osd":6,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.501},"max":{"1min":1.639,"5min":1.671,"15min":1.903},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85599999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2370000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.071}]},{"osd":7,"last update":"Mon Oct 30 12:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"max":{"1min":1.754,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.95499999999999996,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.5920000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.92800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12967936,"data_stored":12941832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42508288,"data_stored":42484041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61579264,"data_stored":61563098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35971072,"data_stored":35950161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":51576832,"data_stored":51557276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264265728,"data_stored":256824371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":262901760,"data_stored":255370516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:58:28.161 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.161 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.162 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.162 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.163 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.164 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.164 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-30T12:45:18.050351+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=45, tm_sec=18, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=12, tm_min=57, tm_sec=41, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T12:58:28.165 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T12:58:46.404 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:58:46.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:58:48.166 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T12:58:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 Processor -- start 2023-10-30T12:58:48.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 -- start start 2023-10-30T12:58:48.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff2200ccf60 0x7ff2200cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2200c2500 0x7ff2200c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff220151aa0 con 0x7ff2201516c0 2023-10-30T12:58:48.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200c74a0 con 0x7ff2200ccf60 2023-10-30T12:58:48.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff225632640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c2e70 con 0x7ff2200c2500 2023-10-30T12:58:48.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff2200ccf60 0x7ff2200cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff2200ccf60 0x7ff2200cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41774/0 (socket says 172.21.15.8:41774) 2023-10-30T12:58:48.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 -- 172.21.15.8:0/212053448 learned_addr learned my addr 172.21.15.8:0/212053448 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:48.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff21effd640 1 --1- 172.21.15.8:0/212053448 >> v1:172.21.15.8:6789/0 conn(0x7ff2201516c0 0x7ff2200c7d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:57752/0 (socket says 172.21.15.8:57752) 2023-10-30T12:58:48.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff21f7fe640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2200c2500 0x7ff2200c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 -- 172.21.15.8:0/212053448 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2200c2500 msgr2=0x7ff2200c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:48.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2200c2500 0x7ff2200c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 -- 172.21.15.8:0/212053448 >> v1:172.21.15.8:6789/0 conn(0x7ff2201516c0 legacy=0x7ff2200c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff21e7fc640 1 -- 172.21.15.8:0/212053448 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106371582 0 0) 0x7ff220151aa0 con 0x7ff2201516c0 2023-10-30T12:58:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.318+0000 7ff217fff640 1 -- 172.21.15.8:0/212053448 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220158af0 con 0x7ff2200ccf60 2023-10-30T12:58:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff217fff640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200ccf60 0x7ff2200cd3a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff208009870 tx=0x7ff20801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39940b8266afd995 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff21e7fc640 1 -- 172.21.15.8:0/212053448 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080ab600 con 0x7ff2200ccf60 2023-10-30T12:58:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff21e7fc640 1 -- 172.21.15.8:0/212053448 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2080a7050 con 0x7ff2200ccf60 2023-10-30T12:58:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff21e7fc640 1 -- 172.21.15.8:0/212053448 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080ab980 con 0x7ff2200ccf60 2023-10-30T12:58:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- 172.21.15.8:0/212053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200ccf60 msgr2=0x7ff2200cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200ccf60 0x7ff2200cd3a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff208009870 tx=0x7ff20801c540 comp rx=0 tx=0).stop 2023-10-30T12:58:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- 172.21.15.8:0/212053448 shutdown_connections 2023-10-30T12:58:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2200c2500 0x7ff2200c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 --2- 172.21.15.8:0/212053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200ccf60 0x7ff2200cd3a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- 172.21.15.8:0/212053448 >> 172.21.15.8:0/212053448 conn(0x7ff220071fb0 msgr2=0x7ff2200723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- 172.21.15.8:0/212053448 shutdown_connections 2023-10-30T12:58:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- 172.21.15.8:0/212053448 wait complete. 2023-10-30T12:58:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 Processor -- start 2023-10-30T12:58:48.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.322+0000 7ff225632640 1 -- start start 2023-10-30T12:58:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2200ccf60 0x7ff220149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2201516c0 0x7ff2200cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220146f80 con 0x7ff2200c2500 2023-10-30T12:58:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201ef9a0 con 0x7ff2200ccf60 2023-10-30T12:58:48.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c8ca0 con 0x7ff2201516c0 2023-10-30T12:58:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41788/0 (socket says 172.21.15.8:41788) 2023-10-30T12:58:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 -- 172.21.15.8:0/1026672552 learned_addr learned my addr 172.21.15.8:0/1026672552 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:48.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21f7fe640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2201516c0 0x7ff2200cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff217fff640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2200ccf60 0x7ff220149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 -- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2201516c0 msgr2=0x7ff2200cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2201516c0 0x7ff2200cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 -- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2200ccf60 msgr2=0x7ff220149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2200ccf60 0x7ff220149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 -- 172.21.15.8:0/1026672552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080094d0 con 0x7ff2200c2500 2023-10-30T12:58:48.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff21effd640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c480 tx=0x7ff21000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2100074e0 con 0x7ff2200c2500 2023-10-30T12:58:48.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210007660 con 0x7ff2200c2500 2023-10-30T12:58:48.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210004a30 con 0x7ff2200c2500 2023-10-30T12:58:48.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2200cc110 con 0x7ff2200c2500 2023-10-30T12:58:48.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2201eff20 con 0x7ff2200c2500 2023-10-30T12:58:48.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2201470f0 con 0x7ff2200c2500 2023-10-30T12:58:48.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.326+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff210004b90 con 0x7ff2200c2500 2023-10-30T12:58:48.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.330+0000 7ff2177fe640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 0x7ff200068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.330+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7ff210083390 con 0x7ff2200c2500 2023-10-30T12:58:48.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.330+0000 7ff217fff640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 0x7ff200068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.330+0000 7ff217fff640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 0x7ff200068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff2200cdb70 tx=0x7ff2080a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.334+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff210053ff0 con 0x7ff2200c2500 2023-10-30T12:58:48.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.486+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 --> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff220157f50 con 0x7ff2000664a0 2023-10-30T12:58:48.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.494+0000 7ff2177fe640 1 -- 172.21.15.8:0/1026672552 <== mgr.4098 v2:172.21.15.8:6832/2382673595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294571 (secure 0 0 0) 0x7ff2080043c0 con 0x7ff2000664a0 2023-10-30T12:58:48.503 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:48.505 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-30T12:58:48.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 msgr2=0x7ff200068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 0x7ff200068960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff2200cdb70 tx=0x7ff2080a6000 comp rx=0 tx=0).stop 2023-10-30T12:58:48.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 msgr2=0x7ff2200c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c480 tx=0x7ff21000c940 comp rx=0 tx=0).stop 2023-10-30T12:58:48.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 shutdown_connections 2023-10-30T12:58:48.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7ff2000664a0 0x7ff200068960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2200ccf60 0x7ff220149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7ff2201516c0 0x7ff2200cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 --2- 172.21.15.8:0/1026672552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2200c2500 0x7ff2200c8760 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 >> 172.21.15.8:0/1026672552 conn(0x7ff220071fb0 msgr2=0x7ff220159940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:48.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 shutdown_connections 2023-10-30T12:58:48.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.498+0000 7ff225632640 1 -- 172.21.15.8:0/1026672552 wait complete. 2023-10-30T12:58:48.523 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2023-10-30T12:58:47.824767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1103643966,"num_objects":10479,"num_object_clones":0,"num_object_copies":20958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10479,"num_whiteouts":0,"num_read":2548,"num_read_kb":243548,"num_write":26760,"num_write_kb":1229594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":692939,"num_omap_keys":1293,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37851,"ondisk_log_size":37851,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2728724,"kb_used_data":2211412,"kb_used_omap":12,"kb_used_meta":517235,"kb_avail":752245996,"statfs":{"total":773094113280,"available":770299899904,"internally_reserved":0,"allocated":2264485888,"data_stored":2208168521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":529649210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003624"},"pg_stats":[{"pgid":"3.3f","version":"53'504","reported_seq":2639,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.025636+0000","last_change":"2023-10-30T12:58:07.025697+0000","last_active":"2023-10-30T12:58:07.025636+0000","last_peered":"2023-10-30T12:58:07.025636+0000","last_clean":"2023-10-30T12:58:07.025636+0000","last_became_active":"2023-10-30T12:45:19.106286+0000","last_became_peered":"2023-10-30T12:45:19.106286+0000","last_unstale":"2023-10-30T12:58:07.025636+0000","last_undegraded":"2023-10-30T12:58:07.025636+0000","last_fullsized":"2023-10-30T12:58:07.025636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'504","last_scrub_stamp":"2023-10-30T12:58:07.025535+0000","last_deep_scrub":"53'504","last_deep_scrub_stamp":"2023-10-30T12:58:07.025535+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.025535+0000","objects_scrubbed":163,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:29:12.472868+0000","scrub_duration":0.24485915899999999,"objects_trimmed":0,"snaptrim_duration":0.007295157,"stat_sum":{"num_bytes":13943365,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":324,"num_write_kb":13742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"53'506","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:21.271626+0000","last_change":"2023-10-30T12:58:21.271719+0000","last_active":"2023-10-30T12:58:21.271626+0000","last_peered":"2023-10-30T12:58:21.271626+0000","last_clean":"2023-10-30T12:58:21.271626+0000","last_became_active":"2023-10-30T12:45:19.085939+0000","last_became_peered":"2023-10-30T12:45:19.085939+0000","last_unstale":"2023-10-30T12:58:21.271626+0000","last_undegraded":"2023-10-30T12:58:21.271626+0000","last_fullsized":"2023-10-30T12:58:21.271626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'506","last_scrub_stamp":"2023-10-30T12:58:21.271522+0000","last_deep_scrub":"53'506","last_deep_scrub_stamp":"2023-10-30T12:58:21.271522+0000","last_clean_scrub_stamp":"2023-10-30T12:58:21.271522+0000","objects_scrubbed":170,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:55:22.196209+0000","scrub_duration":0.153310585,"objects_trimmed":0,"snaptrim_duration":0.002999201,"stat_sum":{"num_bytes":14216413,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":316,"num_write_kb":14055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"53'510","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.591196+0000","last_change":"2023-10-30T12:57:58.591267+0000","last_active":"2023-10-30T12:57:58.591196+0000","last_peered":"2023-10-30T12:57:58.591196+0000","last_clean":"2023-10-30T12:57:58.591196+0000","last_became_active":"2023-10-30T12:45:19.096753+0000","last_became_peered":"2023-10-30T12:45:19.096753+0000","last_unstale":"2023-10-30T12:57:58.591196+0000","last_undegraded":"2023-10-30T12:57:58.591196+0000","last_fullsized":"2023-10-30T12:57:58.591196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'510","last_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_deep_scrub":"53'510","last_deep_scrub_stamp":"2023-10-30T12:57:58.591110+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.591110+0000","objects_scrubbed":152,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:03:49.294530+0000","scrub_duration":0.16537424100000001,"objects_trimmed":0,"snaptrim_duration":0.0033201870000000001,"stat_sum":{"num_bytes":5209367,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":26,"num_read_kb":8209,"num_write":346,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"53'452","reported_seq":2580,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:31.311033+0000","last_change":"2023-10-30T12:58:31.311147+0000","last_active":"2023-10-30T12:58:31.311033+0000","last_peered":"2023-10-30T12:58:31.311033+0000","last_clean":"2023-10-30T12:58:31.311033+0000","last_became_active":"2023-10-30T12:45:19.125411+0000","last_became_peered":"2023-10-30T12:45:19.125411+0000","last_unstale":"2023-10-30T12:58:31.311033+0000","last_undegraded":"2023-10-30T12:58:31.311033+0000","last_fullsized":"2023-10-30T12:58:31.311033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'452","last_scrub_stamp":"2023-10-30T12:58:31.310803+0000","last_deep_scrub":"53'452","last_deep_scrub_stamp":"2023-10-30T12:58:31.310803+0000","last_clean_scrub_stamp":"2023-10-30T12:58:31.310803+0000","objects_scrubbed":151,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:56:03.444744+0000","scrub_duration":0.25600709700000002,"objects_trimmed":0,"snaptrim_duration":0.0068290900000000003,"stat_sum":{"num_bytes":24346225,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":289,"num_write_kb":23925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"53'507","reported_seq":2636,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.102205+0000","last_change":"2023-10-30T12:58:19.102288+0000","last_active":"2023-10-30T12:58:19.102205+0000","last_peered":"2023-10-30T12:58:19.102205+0000","last_clean":"2023-10-30T12:58:19.102205+0000","last_became_active":"2023-10-30T12:45:19.113560+0000","last_became_peered":"2023-10-30T12:45:19.113560+0000","last_unstale":"2023-10-30T12:58:19.102205+0000","last_undegraded":"2023-10-30T12:58:19.102205+0000","last_fullsized":"2023-10-30T12:58:19.102205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-30T12:58:19.102061+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-30T12:58:19.102061+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.102061+0000","objects_scrubbed":156,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:30:09.576623+0000","scrub_duration":0.33207663799999998,"objects_trimmed":0,"snaptrim_duration":0.0069600720000000003,"stat_sum":{"num_bytes":28213093,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":24,"num_read_kb":16,"num_write":332,"num_write_kb":27731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"53'487","reported_seq":2615,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:11.100664+0000","last_change":"2023-10-30T12:58:11.100708+0000","last_active":"2023-10-30T12:58:11.100664+0000","last_peered":"2023-10-30T12:58:11.100664+0000","last_clean":"2023-10-30T12:58:11.100664+0000","last_became_active":"2023-10-30T12:45:19.119218+0000","last_became_peered":"2023-10-30T12:45:19.119218+0000","last_unstale":"2023-10-30T12:58:11.100664+0000","last_undegraded":"2023-10-30T12:58:11.100664+0000","last_fullsized":"2023-10-30T12:58:11.100664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:11.100583+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:11.100583+0000","last_clean_scrub_stamp":"2023-10-30T12:58:11.100583+0000","objects_scrubbed":155,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:33:07.724909+0000","scrub_duration":0.13540437999999999,"objects_trimmed":0,"snaptrim_duration":0.0041663830000000001,"stat_sum":{"num_bytes":10796809,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"53'546","reported_seq":2669,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.122880+0000","last_change":"2023-10-30T12:57:48.122880+0000","last_active":"2023-10-30T12:57:48.122880+0000","last_peered":"2023-10-30T12:57:48.122880+0000","last_clean":"2023-10-30T12:57:48.122880+0000","last_became_active":"2023-10-30T12:45:19.107457+0000","last_became_peered":"2023-10-30T12:45:19.107457+0000","last_unstale":"2023-10-30T12:57:48.122880+0000","last_undegraded":"2023-10-30T12:57:48.122880+0000","last_fullsized":"2023-10-30T12:57:48.122880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'546","last_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_deep_scrub":"53'546","last_deep_scrub_stamp":"2023-10-30T12:57:48.122543+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.122543+0000","objects_scrubbed":180,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:54:00.797869+0000","scrub_duration":0.33340951699999999,"objects_trimmed":0,"snaptrim_duration":0.0054619960000000002,"stat_sum":{"num_bytes":23819931,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":356,"num_write_kb":23455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"53'449","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.579958+0000","last_change":"2023-10-30T12:58:03.580002+0000","last_active":"2023-10-30T12:58:03.579958+0000","last_peered":"2023-10-30T12:58:03.579958+0000","last_clean":"2023-10-30T12:58:03.579958+0000","last_became_active":"2023-10-30T12:45:19.120891+0000","last_became_peered":"2023-10-30T12:45:19.120891+0000","last_unstale":"2023-10-30T12:58:03.579958+0000","last_undegraded":"2023-10-30T12:58:03.579958+0000","last_fullsized":"2023-10-30T12:58:03.579958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-30T12:58:03.579888+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.579888+0000","objects_scrubbed":134,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:08:01.622733+0000","scrub_duration":0.15696447399999999,"objects_trimmed":0,"snaptrim_duration":0.0032684279999999999,"stat_sum":{"num_bytes":14247666,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":28,"num_read_kb":16,"num_write":295,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"53'496","reported_seq":2638,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.995994+0000","last_change":"2023-10-30T12:58:07.996083+0000","last_active":"2023-10-30T12:58:07.995994+0000","last_peered":"2023-10-30T12:58:07.995994+0000","last_clean":"2023-10-30T12:58:07.995994+0000","last_became_active":"2023-10-30T12:45:19.135061+0000","last_became_peered":"2023-10-30T12:45:19.135061+0000","last_unstale":"2023-10-30T12:58:07.995994+0000","last_undegraded":"2023-10-30T12:58:07.995994+0000","last_fullsized":"2023-10-30T12:58:07.995994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:07.995885+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:07.995885+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.995885+0000","objects_scrubbed":159,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:09:06.335910+0000","scrub_duration":0.243363578,"objects_trimmed":0,"snaptrim_duration":0.0073573060000000001,"stat_sum":{"num_bytes":13767488,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":327,"num_write_kb":17709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"53'484","reported_seq":2626,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.665230+0000","last_change":"2023-10-30T12:58:10.665308+0000","last_active":"2023-10-30T12:58:10.665230+0000","last_peered":"2023-10-30T12:58:10.665230+0000","last_clean":"2023-10-30T12:58:10.665230+0000","last_became_active":"2023-10-30T12:45:19.126678+0000","last_became_peered":"2023-10-30T12:45:19.126678+0000","last_unstale":"2023-10-30T12:58:10.665230+0000","last_undegraded":"2023-10-30T12:58:10.665230+0000","last_fullsized":"2023-10-30T12:58:10.665230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-30T12:58:10.665118+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-30T12:58:10.665118+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.665118+0000","objects_scrubbed":145,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:10:19.686235+0000","scrub_duration":0.26126349399999999,"objects_trimmed":0,"snaptrim_duration":0.0061549680000000002,"stat_sum":{"num_bytes":14780154,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":38,"num_read_kb":28,"num_write":323,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"53'500","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.600687+0000","last_change":"2023-10-30T12:58:03.600771+0000","last_active":"2023-10-30T12:58:03.600687+0000","last_peered":"2023-10-30T12:58:03.600687+0000","last_clean":"2023-10-30T12:58:03.600687+0000","last_became_active":"2023-10-30T12:45:19.126683+0000","last_became_peered":"2023-10-30T12:45:19.126683+0000","last_unstale":"2023-10-30T12:58:03.600687+0000","last_undegraded":"2023-10-30T12:58:03.600687+0000","last_fullsized":"2023-10-30T12:58:03.600687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.600594+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.600594+0000","objects_scrubbed":164,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:39:38.231319+0000","scrub_duration":0.225903303,"objects_trimmed":0,"snaptrim_duration":0.0059834689999999999,"stat_sum":{"num_bytes":18469007,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":320,"num_write_kb":18200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"53'459","reported_seq":2586,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.648328+0000","last_change":"2023-10-30T12:57:55.648414+0000","last_active":"2023-10-30T12:57:55.648328+0000","last_peered":"2023-10-30T12:57:55.648328+0000","last_clean":"2023-10-30T12:57:55.648328+0000","last_became_active":"2023-10-30T12:45:19.096517+0000","last_became_peered":"2023-10-30T12:45:19.096517+0000","last_unstale":"2023-10-30T12:57:55.648328+0000","last_undegraded":"2023-10-30T12:57:55.648328+0000","last_fullsized":"2023-10-30T12:57:55.648328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'459","last_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_deep_scrub":"53'459","last_deep_scrub_stamp":"2023-10-30T12:57:55.648231+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.648231+0000","objects_scrubbed":147,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:45:01.167844+0000","scrub_duration":0.24832579699999999,"objects_trimmed":0,"snaptrim_duration":0.0025618469999999999,"stat_sum":{"num_bytes":22259614,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":295,"num_write_kb":25988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"53'477","reported_seq":2616,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:17.669645+0000","last_change":"2023-10-30T12:58:17.669698+0000","last_active":"2023-10-30T12:58:17.669645+0000","last_peered":"2023-10-30T12:58:17.669645+0000","last_clean":"2023-10-30T12:58:17.669645+0000","last_became_active":"2023-10-30T12:45:19.127704+0000","last_became_peered":"2023-10-30T12:45:19.127704+0000","last_unstale":"2023-10-30T12:58:17.669645+0000","last_undegraded":"2023-10-30T12:58:17.669645+0000","last_fullsized":"2023-10-30T12:58:17.669645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-30T12:58:17.669571+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-30T12:58:17.669571+0000","last_clean_scrub_stamp":"2023-10-30T12:58:17.669571+0000","objects_scrubbed":163,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:00:48.499603+0000","scrub_duration":0.21456085499999999,"objects_trimmed":0,"snaptrim_duration":0.0027209980000000001,"stat_sum":{"num_bytes":14858922,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":34,"num_read_kb":4119,"num_write":303,"num_write_kb":22859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"53'500","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.561433+0000","last_change":"2023-10-30T12:58:03.561531+0000","last_active":"2023-10-30T12:58:03.561433+0000","last_peered":"2023-10-30T12:58:03.561433+0000","last_clean":"2023-10-30T12:58:03.561433+0000","last_became_active":"2023-10-30T12:45:19.096493+0000","last_became_peered":"2023-10-30T12:45:19.096493+0000","last_unstale":"2023-10-30T12:58:03.561433+0000","last_undegraded":"2023-10-30T12:58:03.561433+0000","last_fullsized":"2023-10-30T12:58:03.561433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'500","last_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_deep_scrub":"53'500","last_deep_scrub_stamp":"2023-10-30T12:58:03.561319+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.561319+0000","objects_scrubbed":148,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:27.469036+0000","scrub_duration":0.14169902500000001,"objects_trimmed":0,"snaptrim_duration":0.0035593389999999999,"stat_sum":{"num_bytes":18225844,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":25,"num_read_kb":17,"num_write":338,"num_write_kb":17975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"53'510","reported_seq":2641,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.347606+0000","last_change":"2023-10-30T12:58:19.347670+0000","last_active":"2023-10-30T12:58:19.347606+0000","last_peered":"2023-10-30T12:58:19.347606+0000","last_clean":"2023-10-30T12:58:19.347606+0000","last_became_active":"2023-10-30T12:45:19.094180+0000","last_became_peered":"2023-10-30T12:45:19.094180+0000","last_unstale":"2023-10-30T12:58:19.347606+0000","last_undegraded":"2023-10-30T12:58:19.347606+0000","last_fullsized":"2023-10-30T12:58:19.347606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'510","last_scrub_stamp":"2023-10-30T12:58:19.347521+0000","last_deep_scrub":"53'510","last_deep_scrub_stamp":"2023-10-30T12:58:19.347521+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.347521+0000","objects_scrubbed":166,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:01:18.461141+0000","scrub_duration":0.21695156800000001,"objects_trimmed":0,"snaptrim_duration":0.00242932,"stat_sum":{"num_bytes":12806578,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":26,"num_read_kb":4114,"num_write":328,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"53'535","reported_seq":2673,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.149981+0000","last_change":"2023-10-30T12:58:09.150069+0000","last_active":"2023-10-30T12:58:09.149981+0000","last_peered":"2023-10-30T12:58:09.149981+0000","last_clean":"2023-10-30T12:58:09.149981+0000","last_became_active":"2023-10-30T12:45:19.107506+0000","last_became_peered":"2023-10-30T12:45:19.107506+0000","last_unstale":"2023-10-30T12:58:09.149981+0000","last_undegraded":"2023-10-30T12:58:09.149981+0000","last_fullsized":"2023-10-30T12:58:09.149981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'535","last_scrub_stamp":"2023-10-30T12:58:09.149880+0000","last_deep_scrub":"53'535","last_deep_scrub_stamp":"2023-10-30T12:58:09.149880+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.149880+0000","objects_scrubbed":166,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:57:22.762278+0000","scrub_duration":0.178880715,"objects_trimmed":0,"snaptrim_duration":0.0044982110000000002,"stat_sum":{"num_bytes":23758433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":33,"num_read_kb":18,"num_write":359,"num_write_kb":26630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"53'493","reported_seq":2626,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:33.347673+0000","last_change":"2023-10-30T12:58:33.347722+0000","last_active":"2023-10-30T12:58:33.347673+0000","last_peered":"2023-10-30T12:58:33.347673+0000","last_clean":"2023-10-30T12:58:33.347673+0000","last_became_active":"2023-10-30T12:45:19.123947+0000","last_became_peered":"2023-10-30T12:45:19.123947+0000","last_unstale":"2023-10-30T12:58:33.347673+0000","last_undegraded":"2023-10-30T12:58:33.347673+0000","last_fullsized":"2023-10-30T12:58:33.347673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-30T12:58:33.347586+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-30T12:58:33.347586+0000","last_clean_scrub_stamp":"2023-10-30T12:58:33.347586+0000","objects_scrubbed":155,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:26:16.483793+0000","scrub_duration":0.236927165,"objects_trimmed":0,"snaptrim_duration":0.00762629,"stat_sum":{"num_bytes":10013359,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":28,"num_read_kb":8209,"num_write":323,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"53'508","reported_seq":2652,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:12.252082+0000","last_change":"2023-10-30T12:58:12.252141+0000","last_active":"2023-10-30T12:58:12.252082+0000","last_peered":"2023-10-30T12:58:12.252082+0000","last_clean":"2023-10-30T12:58:12.252082+0000","last_became_active":"2023-10-30T12:45:19.096074+0000","last_became_peered":"2023-10-30T12:45:19.096074+0000","last_unstale":"2023-10-30T12:58:12.252082+0000","last_undegraded":"2023-10-30T12:58:12.252082+0000","last_fullsized":"2023-10-30T12:58:12.252082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2023-10-30T12:58:12.252006+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2023-10-30T12:58:12.252006+0000","last_clean_scrub_stamp":"2023-10-30T12:58:12.252006+0000","objects_scrubbed":162,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:56:22.896888+0000","scrub_duration":0.22659649400000001,"objects_trimmed":0,"snaptrim_duration":0.0015703519999999999,"stat_sum":{"num_bytes":10354437,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":330,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"53'524","reported_seq":2651,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.142463+0000","last_change":"2023-10-30T12:58:22.142570+0000","last_active":"2023-10-30T12:58:22.142463+0000","last_peered":"2023-10-30T12:58:22.142463+0000","last_clean":"2023-10-30T12:58:22.142463+0000","last_became_active":"2023-10-30T12:45:19.106619+0000","last_became_peered":"2023-10-30T12:45:19.106619+0000","last_unstale":"2023-10-30T12:58:22.142463+0000","last_undegraded":"2023-10-30T12:58:22.142463+0000","last_fullsized":"2023-10-30T12:58:22.142463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'524","last_scrub_stamp":"2023-10-30T12:58:22.142304+0000","last_deep_scrub":"53'524","last_deep_scrub_stamp":"2023-10-30T12:58:22.142304+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.142304+0000","objects_scrubbed":163,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:24:29.904983+0000","scrub_duration":0.31798768799999999,"objects_trimmed":0,"snaptrim_duration":0.006868089,"stat_sum":{"num_bytes":14695113,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":22,"num_read_kb":4113,"num_write":341,"num_write_kb":18629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"53'507","reported_seq":2639,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.614493+0000","last_change":"2023-10-30T12:58:13.614550+0000","last_active":"2023-10-30T12:58:13.614493+0000","last_peered":"2023-10-30T12:58:13.614493+0000","last_clean":"2023-10-30T12:58:13.614493+0000","last_became_active":"2023-10-30T12:45:19.097218+0000","last_became_peered":"2023-10-30T12:45:19.097218+0000","last_unstale":"2023-10-30T12:58:13.614493+0000","last_undegraded":"2023-10-30T12:58:13.614493+0000","last_fullsized":"2023-10-30T12:58:13.614493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-30T12:58:13.614391+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-30T12:58:13.614391+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.614391+0000","objects_scrubbed":156,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:07:20.070473+0000","scrub_duration":0.212317802,"objects_trimmed":0,"snaptrim_duration":0.0060608179999999999,"stat_sum":{"num_bytes":9489835,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":4113,"num_write":336,"num_write_kb":17636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"53'517","reported_seq":2656,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:18.634707+0000","last_change":"2023-10-30T12:58:18.634758+0000","last_active":"2023-10-30T12:58:18.634707+0000","last_peered":"2023-10-30T12:58:18.634707+0000","last_clean":"2023-10-30T12:58:18.634707+0000","last_became_active":"2023-10-30T12:45:19.120105+0000","last_became_peered":"2023-10-30T12:45:19.120105+0000","last_unstale":"2023-10-30T12:58:18.634707+0000","last_undegraded":"2023-10-30T12:58:18.634707+0000","last_fullsized":"2023-10-30T12:58:18.634707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'517","last_scrub_stamp":"2023-10-30T12:58:18.634630+0000","last_deep_scrub":"53'517","last_deep_scrub_stamp":"2023-10-30T12:58:18.634630+0000","last_clean_scrub_stamp":"2023-10-30T12:58:18.634630+0000","objects_scrubbed":162,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:20:03.977601+0000","scrub_duration":0.23488492599999999,"objects_trimmed":0,"snaptrim_duration":0.002440663,"stat_sum":{"num_bytes":2661743,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":34,"num_read_kb":21,"num_write":342,"num_write_kb":2703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"53'561","reported_seq":2700,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:16.293081+0000","last_change":"2023-10-30T12:58:16.293128+0000","last_active":"2023-10-30T12:58:16.293081+0000","last_peered":"2023-10-30T12:58:16.293081+0000","last_clean":"2023-10-30T12:58:16.293081+0000","last_became_active":"2023-10-30T12:45:59.303455+0000","last_became_peered":"2023-10-30T12:45:59.303455+0000","last_unstale":"2023-10-30T12:58:16.293081+0000","last_undegraded":"2023-10-30T12:58:16.293081+0000","last_fullsized":"2023-10-30T12:58:16.293081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'561","last_scrub_stamp":"2023-10-30T12:58:16.292999+0000","last_deep_scrub":"53'561","last_deep_scrub_stamp":"2023-10-30T12:58:16.292999+0000","last_clean_scrub_stamp":"2023-10-30T12:58:16.292999+0000","objects_scrubbed":178,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:58:07.559376+0000","scrub_duration":0.26458689699999999,"objects_trimmed":0,"snaptrim_duration":0.001611517,"stat_sum":{"num_bytes":15579086,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":37,"num_read_kb":4118,"num_write":363,"num_write_kb":19496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"53'533","reported_seq":2677,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.612587+0000","last_change":"2023-10-30T12:58:01.612683+0000","last_active":"2023-10-30T12:58:01.612587+0000","last_peered":"2023-10-30T12:58:01.612587+0000","last_clean":"2023-10-30T12:58:01.612587+0000","last_became_active":"2023-10-30T12:45:19.094459+0000","last_became_peered":"2023-10-30T12:45:19.094459+0000","last_unstale":"2023-10-30T12:58:01.612587+0000","last_undegraded":"2023-10-30T12:58:01.612587+0000","last_fullsized":"2023-10-30T12:58:01.612587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'533","last_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_deep_scrub":"53'533","last_deep_scrub_stamp":"2023-10-30T12:58:01.612496+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.612496+0000","objects_scrubbed":161,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:28:32.894742+0000","scrub_duration":0.16518128200000001,"objects_trimmed":0,"snaptrim_duration":0.0025976850000000002,"stat_sum":{"num_bytes":16035861,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":39,"num_read_kb":8213,"num_write":353,"num_write_kb":19942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"53'534","reported_seq":2662,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:03.217312+0000","last_change":"2023-10-30T12:58:03.217312+0000","last_active":"2023-10-30T12:58:03.217312+0000","last_peered":"2023-10-30T12:58:03.217312+0000","last_clean":"2023-10-30T12:58:03.217312+0000","last_became_active":"2023-10-30T12:45:19.106700+0000","last_became_peered":"2023-10-30T12:45:19.106700+0000","last_unstale":"2023-10-30T12:58:03.217312+0000","last_undegraded":"2023-10-30T12:58:03.217312+0000","last_fullsized":"2023-10-30T12:58:03.217312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'534","last_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_deep_scrub":"53'534","last_deep_scrub_stamp":"2023-10-30T12:58:03.216977+0000","last_clean_scrub_stamp":"2023-10-30T12:58:03.216977+0000","objects_scrubbed":173,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:50:07.436714+0000","scrub_duration":0.30228984199999998,"objects_trimmed":0,"snaptrim_duration":0.0058847760000000004,"stat_sum":{"num_bytes":19277566,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":341,"num_write_kb":19009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"53'495","reported_seq":2624,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:12.715377+0000","last_change":"2023-10-30T12:58:12.715377+0000","last_active":"2023-10-30T12:58:12.715377+0000","last_peered":"2023-10-30T12:58:12.715377+0000","last_clean":"2023-10-30T12:58:12.715377+0000","last_became_active":"2023-10-30T12:45:19.098392+0000","last_became_peered":"2023-10-30T12:45:19.098392+0000","last_unstale":"2023-10-30T12:58:12.715377+0000","last_undegraded":"2023-10-30T12:58:12.715377+0000","last_fullsized":"2023-10-30T12:58:12.715377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:58:12.715237+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:58:12.715237+0000","last_clean_scrub_stamp":"2023-10-30T12:58:12.715237+0000","objects_scrubbed":168,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:51:52.292380+0000","scrub_duration":0.228241844,"objects_trimmed":0,"snaptrim_duration":0.0029594920000000002,"stat_sum":{"num_bytes":23544891,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":23,"num_read_kb":12,"num_write":316,"num_write_kb":23165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"53'545","reported_seq":2692,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:14.584433+0000","last_change":"2023-10-30T12:58:14.584433+0000","last_active":"2023-10-30T12:58:14.584433+0000","last_peered":"2023-10-30T12:58:14.584433+0000","last_clean":"2023-10-30T12:58:14.584433+0000","last_became_active":"2023-10-30T12:45:19.110415+0000","last_became_peered":"2023-10-30T12:45:19.110415+0000","last_unstale":"2023-10-30T12:58:14.584433+0000","last_undegraded":"2023-10-30T12:58:14.584433+0000","last_fullsized":"2023-10-30T12:58:14.584433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'545","last_scrub_stamp":"2023-10-30T12:58:14.584283+0000","last_deep_scrub":"53'545","last_deep_scrub_stamp":"2023-10-30T12:58:14.584283+0000","last_clean_scrub_stamp":"2023-10-30T12:58:14.584283+0000","objects_scrubbed":166,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:37:08.795274+0000","scrub_duration":0.223781852,"objects_trimmed":0,"snaptrim_duration":0.0059470790000000001,"stat_sum":{"num_bytes":7055596,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":41,"num_read_kb":24,"num_write":350,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"53'493","reported_seq":2623,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:26.377866+0000","last_change":"2023-10-30T12:58:26.377955+0000","last_active":"2023-10-30T12:58:26.377866+0000","last_peered":"2023-10-30T12:58:26.377866+0000","last_clean":"2023-10-30T12:58:26.377866+0000","last_became_active":"2023-10-30T12:45:19.080006+0000","last_became_peered":"2023-10-30T12:45:19.080006+0000","last_unstale":"2023-10-30T12:58:26.377866+0000","last_undegraded":"2023-10-30T12:58:26.377866+0000","last_fullsized":"2023-10-30T12:58:26.377866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-30T12:58:26.377776+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-30T12:58:26.377776+0000","last_clean_scrub_stamp":"2023-10-30T12:58:26.377776+0000","objects_scrubbed":165,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:42:57.426355+0000","scrub_duration":0.15521407300000001,"objects_trimmed":0,"snaptrim_duration":0.0027520499999999998,"stat_sum":{"num_bytes":22126442,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":24,"num_read_kb":4111,"num_write":313,"num_write_kb":21771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"53'433","reported_seq":2561,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.668381+0000","last_change":"2023-10-30T12:58:22.668425+0000","last_active":"2023-10-30T12:58:22.668381+0000","last_peered":"2023-10-30T12:58:22.668381+0000","last_clean":"2023-10-30T12:58:22.668381+0000","last_became_active":"2023-10-30T12:45:19.094221+0000","last_became_peered":"2023-10-30T12:45:19.094221+0000","last_unstale":"2023-10-30T12:58:22.668381+0000","last_undegraded":"2023-10-30T12:58:22.668381+0000","last_fullsized":"2023-10-30T12:58:22.668381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'433","last_scrub_stamp":"2023-10-30T12:58:22.668305+0000","last_deep_scrub":"53'433","last_deep_scrub_stamp":"2023-10-30T12:58:22.668305+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.668305+0000","objects_scrubbed":127,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:09:21.611583+0000","scrub_duration":0.16211444699999999,"objects_trimmed":0,"snaptrim_duration":0.0030313829999999999,"stat_sum":{"num_bytes":6280955,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":23,"num_read_kb":3284,"num_write":287,"num_write_kb":9554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"53'481","reported_seq":2612,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.751923+0000","last_change":"2023-10-30T12:57:49.751923+0000","last_active":"2023-10-30T12:57:49.751923+0000","last_peered":"2023-10-30T12:57:49.751923+0000","last_clean":"2023-10-30T12:57:49.751923+0000","last_became_active":"2023-10-30T12:45:19.092720+0000","last_became_peered":"2023-10-30T12:45:19.092720+0000","last_unstale":"2023-10-30T12:57:49.751923+0000","last_undegraded":"2023-10-30T12:57:49.751923+0000","last_fullsized":"2023-10-30T12:57:49.751923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'481","last_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_deep_scrub":"53'481","last_deep_scrub_stamp":"2023-10-30T12:57:49.751798+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.751798+0000","objects_scrubbed":156,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:02:28.001220+0000","scrub_duration":0.14818989499999999,"objects_trimmed":0,"snaptrim_duration":0.003397981,"stat_sum":{"num_bytes":18993247,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":25,"num_read_kb":14,"num_write":315,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"53'555","reported_seq":2692,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.242980+0000","last_change":"2023-10-30T12:58:10.243033+0000","last_active":"2023-10-30T12:58:10.242980+0000","last_peered":"2023-10-30T12:58:10.242980+0000","last_clean":"2023-10-30T12:58:10.242980+0000","last_became_active":"2023-10-30T12:45:19.106363+0000","last_became_peered":"2023-10-30T12:45:19.106363+0000","last_unstale":"2023-10-30T12:58:10.242980+0000","last_undegraded":"2023-10-30T12:58:10.242980+0000","last_fullsized":"2023-10-30T12:58:10.242980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'555","last_scrub_stamp":"2023-10-30T12:58:10.242873+0000","last_deep_scrub":"53'555","last_deep_scrub_stamp":"2023-10-30T12:58:10.242873+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.242873+0000","objects_scrubbed":180,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:28:57.920616+0000","scrub_duration":0.241089517,"objects_trimmed":0,"snaptrim_duration":0.0061990070000000003,"stat_sum":{"num_bytes":15318011,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":30,"num_read_kb":20,"num_write":358,"num_write_kb":15146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"53'551","reported_seq":2689,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.240366+0000","last_change":"2023-10-30T12:58:10.240430+0000","last_active":"2023-10-30T12:58:10.240366+0000","last_peered":"2023-10-30T12:58:10.240366+0000","last_clean":"2023-10-30T12:58:10.240366+0000","last_became_active":"2023-10-30T12:45:19.107265+0000","last_became_peered":"2023-10-30T12:45:19.107265+0000","last_unstale":"2023-10-30T12:58:10.240366+0000","last_undegraded":"2023-10-30T12:58:10.240366+0000","last_fullsized":"2023-10-30T12:58:10.240366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'551","last_scrub_stamp":"2023-10-30T12:58:10.240254+0000","last_deep_scrub":"53'551","last_deep_scrub_stamp":"2023-10-30T12:58:10.240254+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.240254+0000","objects_scrubbed":175,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:54:53.225603+0000","scrub_duration":0.169709203,"objects_trimmed":0,"snaptrim_duration":0.0016857759999999999,"stat_sum":{"num_bytes":9946813,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":31,"num_read_kb":22,"num_write":363,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"53'476","reported_seq":2608,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:04.621477+0000","last_change":"2023-10-30T12:58:04.621599+0000","last_active":"2023-10-30T12:58:04.621477+0000","last_peered":"2023-10-30T12:58:04.621477+0000","last_clean":"2023-10-30T12:58:04.621477+0000","last_became_active":"2023-10-30T12:45:19.087271+0000","last_became_peered":"2023-10-30T12:45:19.087271+0000","last_unstale":"2023-10-30T12:58:04.621477+0000","last_undegraded":"2023-10-30T12:58:04.621477+0000","last_fullsized":"2023-10-30T12:58:04.621477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2023-10-30T12:58:04.621266+0000","last_clean_scrub_stamp":"2023-10-30T12:58:04.621266+0000","objects_scrubbed":140,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:22:51.575535+0000","scrub_duration":0.236240279,"objects_trimmed":0,"snaptrim_duration":0.0032047460000000001,"stat_sum":{"num_bytes":5491481,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":27,"num_read_kb":20,"num_write":324,"num_write_kb":5534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"53'492","reported_seq":2632,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.717128+0000","last_change":"2023-10-30T12:57:50.717216+0000","last_active":"2023-10-30T12:57:50.717128+0000","last_peered":"2023-10-30T12:57:50.717128+0000","last_clean":"2023-10-30T12:57:50.717128+0000","last_became_active":"2023-10-30T12:45:19.092789+0000","last_became_peered":"2023-10-30T12:45:19.092789+0000","last_unstale":"2023-10-30T12:57:50.717128+0000","last_undegraded":"2023-10-30T12:57:50.717128+0000","last_fullsized":"2023-10-30T12:57:50.717128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'492","last_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_deep_scrub":"53'492","last_deep_scrub_stamp":"2023-10-30T12:57:50.717022+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.717022+0000","objects_scrubbed":158,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:25:37.999028+0000","scrub_duration":0.26506830399999998,"objects_trimmed":0,"snaptrim_duration":0.0030563869999999998,"stat_sum":{"num_bytes":14492800,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":35,"num_read_kb":17,"num_write":324,"num_write_kb":14325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"53'267","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.465321+0000","last_change":"2023-10-30T12:57:48.465321+0000","last_active":"2023-10-30T12:57:48.465321+0000","last_peered":"2023-10-30T12:57:48.465321+0000","last_clean":"2023-10-30T12:57:48.465321+0000","last_became_active":"2023-10-30T12:45:18.114108+0000","last_became_peered":"2023-10-30T12:45:18.114108+0000","last_unstale":"2023-10-30T12:57:48.465321+0000","last_undegraded":"2023-10-30T12:57:48.465321+0000","last_fullsized":"2023-10-30T12:57:48.465321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'267","last_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_deep_scrub":"53'267","last_deep_scrub_stamp":"2023-10-30T12:57:48.465192+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.465192+0000","objects_scrubbed":5,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:20:16.586033+0000","scrub_duration":0.036637109000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":20482,"num_write":259,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8500,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"53'466","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:46.810452+0000","last_change":"2023-10-30T12:57:46.810452+0000","last_active":"2023-10-30T12:57:46.810452+0000","last_peered":"2023-10-30T12:57:46.810452+0000","last_clean":"2023-10-30T12:57:46.810452+0000","last_became_active":"2023-10-30T12:45:19.121677+0000","last_became_peered":"2023-10-30T12:45:19.121677+0000","last_unstale":"2023-10-30T12:57:46.810452+0000","last_undegraded":"2023-10-30T12:57:46.810452+0000","last_fullsized":"2023-10-30T12:57:46.810452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-30T12:57:46.810303+0000","last_clean_scrub_stamp":"2023-10-30T12:57:46.810303+0000","objects_scrubbed":147,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:25:30.111932+0000","scrub_duration":0.192780493,"objects_trimmed":0,"snaptrim_duration":0.003615774,"stat_sum":{"num_bytes":7738550,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":315,"num_write_kb":7722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"53'318","reported_seq":452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:25.294029+0000","last_change":"2023-10-30T12:57:57.478006+0000","last_active":"2023-10-30T12:58:25.294029+0000","last_peered":"2023-10-30T12:58:25.294029+0000","last_clean":"2023-10-30T12:58:25.294029+0000","last_became_active":"2023-10-30T12:45:18.101828+0000","last_became_peered":"2023-10-30T12:45:18.101828+0000","last_unstale":"2023-10-30T12:58:25.294029+0000","last_undegraded":"2023-10-30T12:58:25.294029+0000","last_fullsized":"2023-10-30T12:58:25.294029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'311","last_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_deep_scrub":"53'311","last_deep_scrub_stamp":"2023-10-30T12:57:57.477645+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.477645+0000","objects_scrubbed":9,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:49:22.878603+0000","scrub_duration":0.042990041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11333518,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":87,"num_read_kb":8499,"num_write":307,"num_write_kb":11297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":49566,"num_omap_keys":93,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"53'74","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.463815+0000","last_change":"2023-10-30T12:57:49.463904+0000","last_active":"2023-10-30T12:57:49.463815+0000","last_peered":"2023-10-30T12:57:49.463815+0000","last_clean":"2023-10-30T12:57:49.463815+0000","last_became_active":"2023-10-30T12:45:18.097984+0000","last_became_peered":"2023-10-30T12:45:18.097984+0000","last_unstale":"2023-10-30T12:57:49.463815+0000","last_undegraded":"2023-10-30T12:57:49.463815+0000","last_fullsized":"2023-10-30T12:57:49.463815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'74","last_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_deep_scrub":"53'74","last_deep_scrub_stamp":"2023-10-30T12:57:49.463707+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.463707+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:46:36.321809+0000","scrub_duration":0.0076797799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182103,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":242,"num_write":67,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"53'574","reported_seq":2721,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.610591+0000","last_change":"2023-10-30T12:57:58.610634+0000","last_active":"2023-10-30T12:57:58.610591+0000","last_peered":"2023-10-30T12:57:58.610591+0000","last_clean":"2023-10-30T12:57:58.610591+0000","last_became_active":"2023-10-30T12:45:19.088481+0000","last_became_peered":"2023-10-30T12:45:19.088481+0000","last_unstale":"2023-10-30T12:57:58.610591+0000","last_undegraded":"2023-10-30T12:57:58.610591+0000","last_fullsized":"2023-10-30T12:57:58.610591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'574","last_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_deep_scrub":"53'574","last_deep_scrub_stamp":"2023-10-30T12:57:58.610507+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.610507+0000","objects_scrubbed":176,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:26:34.361815+0000","scrub_duration":0.18830925700000001,"objects_trimmed":0,"snaptrim_duration":0.0033321940000000001,"stat_sum":{"num_bytes":10022763,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":41,"num_read_kb":4120,"num_write":382,"num_write_kb":18182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"53'347","reported_seq":407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.875242+0000","last_change":"2023-10-30T12:57:45.875338+0000","last_active":"2023-10-30T12:57:45.875242+0000","last_peered":"2023-10-30T12:57:45.875242+0000","last_clean":"2023-10-30T12:57:45.875242+0000","last_became_active":"2023-10-30T12:45:18.110919+0000","last_became_peered":"2023-10-30T12:45:18.110919+0000","last_unstale":"2023-10-30T12:57:45.875242+0000","last_undegraded":"2023-10-30T12:57:45.875242+0000","last_fullsized":"2023-10-30T12:57:45.875242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'347","last_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_deep_scrub":"53'347","last_deep_scrub_stamp":"2023-10-30T12:57:45.875104+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.875104+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:05.779308+0000","scrub_duration":0.036461027999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":337,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37674,"num_omap_keys":69,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"53'540","reported_seq":2677,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:23.374500+0000","last_change":"2023-10-30T12:58:23.374581+0000","last_active":"2023-10-30T12:58:23.374500+0000","last_peered":"2023-10-30T12:58:23.374500+0000","last_clean":"2023-10-30T12:58:23.374500+0000","last_became_active":"2023-10-30T12:45:19.079348+0000","last_became_peered":"2023-10-30T12:45:19.079348+0000","last_unstale":"2023-10-30T12:58:23.374500+0000","last_undegraded":"2023-10-30T12:58:23.374500+0000","last_fullsized":"2023-10-30T12:58:23.374500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'540","last_scrub_stamp":"2023-10-30T12:58:23.374410+0000","last_deep_scrub":"53'540","last_deep_scrub_stamp":"2023-10-30T12:58:23.374410+0000","last_clean_scrub_stamp":"2023-10-30T12:58:23.374410+0000","objects_scrubbed":184,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:22:34.309284+0000","scrub_duration":0.167791421,"objects_trimmed":0,"snaptrim_duration":0.0021825249999999998,"stat_sum":{"num_bytes":18845974,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":31,"num_read_kb":12308,"num_write":342,"num_write_kb":22679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"53'36","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:05.287943+0000","last_change":"2023-10-30T12:57:58.423647+0000","last_active":"2023-10-30T12:58:05.287943+0000","last_peered":"2023-10-30T12:58:05.287943+0000","last_clean":"2023-10-30T12:58:05.287943+0000","last_became_active":"2023-10-30T12:45:18.107686+0000","last_became_peered":"2023-10-30T12:45:18.107686+0000","last_unstale":"2023-10-30T12:58:05.287943+0000","last_undegraded":"2023-10-30T12:58:05.287943+0000","last_fullsized":"2023-10-30T12:58:05.287943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2023-10-30T12:57:58.423320+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.423320+0000","objects_scrubbed":3,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:20:35.335688+0000","scrub_duration":0.0090485700000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":34,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9500,"num_omap_keys":19,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"53'518","reported_seq":2657,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:32.305005+0000","last_change":"2023-10-30T12:58:32.305050+0000","last_active":"2023-10-30T12:58:32.305005+0000","last_peered":"2023-10-30T12:58:32.305005+0000","last_clean":"2023-10-30T12:58:32.305005+0000","last_became_active":"2023-10-30T12:45:19.121638+0000","last_became_peered":"2023-10-30T12:45:19.121638+0000","last_unstale":"2023-10-30T12:58:32.305005+0000","last_undegraded":"2023-10-30T12:58:32.305005+0000","last_fullsized":"2023-10-30T12:58:32.305005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'518","last_scrub_stamp":"2023-10-30T12:58:32.304914+0000","last_deep_scrub":"53'518","last_deep_scrub_stamp":"2023-10-30T12:58:32.304914+0000","last_clean_scrub_stamp":"2023-10-30T12:58:32.304914+0000","objects_scrubbed":151,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:25:28.619538+0000","scrub_duration":0.24246099099999999,"objects_trimmed":0,"snaptrim_duration":0.0069629219999999999,"stat_sum":{"num_bytes":6175958,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":349,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"53'357","reported_seq":420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:35.290783+0000","last_change":"2023-10-30T12:57:57.930204+0000","last_active":"2023-10-30T12:58:35.290783+0000","last_peered":"2023-10-30T12:58:35.290783+0000","last_clean":"2023-10-30T12:58:35.290783+0000","last_became_active":"2023-10-30T12:45:18.102398+0000","last_became_peered":"2023-10-30T12:45:18.102398+0000","last_unstale":"2023-10-30T12:58:35.290783+0000","last_undegraded":"2023-10-30T12:58:35.290783+0000","last_fullsized":"2023-10-30T12:58:35.290783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'335","last_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_deep_scrub":"53'335","last_deep_scrub_stamp":"2023-10-30T12:57:57.929845+0000","last_clean_scrub_stamp":"2023-10-30T12:57:57.929845+0000","objects_scrubbed":4,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:39:04.674566+0000","scrub_duration":0.019613695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2409933,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":1051,"num_write":353,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26754,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"53'519","reported_seq":2662,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.566278+0000","last_change":"2023-10-30T12:58:00.566341+0000","last_active":"2023-10-30T12:58:00.566278+0000","last_peered":"2023-10-30T12:58:00.566278+0000","last_clean":"2023-10-30T12:58:00.566278+0000","last_became_active":"2023-10-30T12:45:59.540523+0000","last_became_peered":"2023-10-30T12:45:59.540523+0000","last_unstale":"2023-10-30T12:58:00.566278+0000","last_undegraded":"2023-10-30T12:58:00.566278+0000","last_fullsized":"2023-10-30T12:58:00.566278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'519","last_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_deep_scrub":"53'519","last_deep_scrub_stamp":"2023-10-30T12:58:00.566191+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.566191+0000","objects_scrubbed":170,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:02:51.046360+0000","scrub_duration":0.153203224,"objects_trimmed":0,"snaptrim_duration":0.0033800050000000002,"stat_sum":{"num_bytes":14524058,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":31,"num_read_kb":19,"num_write":330,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"53'49","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.101254+0000","last_change":"2023-10-30T12:57:52.101317+0000","last_active":"2023-10-30T12:57:52.101254+0000","last_peered":"2023-10-30T12:57:52.101254+0000","last_clean":"2023-10-30T12:57:52.101254+0000","last_became_active":"2023-10-30T12:45:18.088788+0000","last_became_peered":"2023-10-30T12:45:18.088788+0000","last_unstale":"2023-10-30T12:57:52.101254+0000","last_undegraded":"2023-10-30T12:57:52.101254+0000","last_fullsized":"2023-10-30T12:57:52.101254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'49","last_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_deep_scrub":"53'49","last_deep_scrub_stamp":"2023-10-30T12:57:52.101149+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.101149+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:35:45.571818+0000","scrub_duration":0.011136024,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":46,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":26812,"num_omap_keys":51,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'498","reported_seq":2627,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.764619+0000","last_change":"2023-10-30T12:57:53.764679+0000","last_active":"2023-10-30T12:57:53.764619+0000","last_peered":"2023-10-30T12:57:53.764619+0000","last_clean":"2023-10-30T12:57:53.764619+0000","last_became_active":"2023-10-30T12:45:19.115905+0000","last_became_peered":"2023-10-30T12:45:19.115905+0000","last_unstale":"2023-10-30T12:57:53.764619+0000","last_undegraded":"2023-10-30T12:57:53.764619+0000","last_fullsized":"2023-10-30T12:57:53.764619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-30T12:57:53.764533+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.764533+0000","objects_scrubbed":168,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:20:27.949277+0000","scrub_duration":0.31117073299999998,"objects_trimmed":0,"snaptrim_duration":0.0030461239999999999,"stat_sum":{"num_bytes":10402121,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":24,"num_read_kb":17,"num_write":319,"num_write_kb":14375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"53'115","reported_seq":181,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:05.290301+0000","last_change":"2023-10-30T12:57:48.479616+0000","last_active":"2023-10-30T12:58:05.290301+0000","last_peered":"2023-10-30T12:58:05.290301+0000","last_clean":"2023-10-30T12:58:05.290301+0000","last_became_active":"2023-10-30T12:45:18.083778+0000","last_became_peered":"2023-10-30T12:45:18.083778+0000","last_unstale":"2023-10-30T12:58:05.290301+0000","last_undegraded":"2023-10-30T12:58:05.290301+0000","last_fullsized":"2023-10-30T12:58:05.290301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-30T12:57:48.479247+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.479247+0000","objects_scrubbed":6,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:43:36.989062+0000","scrub_duration":0.019741666000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":109,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8868,"num_omap_keys":17,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"53'472","reported_seq":2592,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.006059+0000","last_change":"2023-10-30T12:58:13.006188+0000","last_active":"2023-10-30T12:58:13.006059+0000","last_peered":"2023-10-30T12:58:13.006059+0000","last_clean":"2023-10-30T12:58:13.006059+0000","last_became_active":"2023-10-30T12:45:19.106578+0000","last_became_peered":"2023-10-30T12:45:19.106578+0000","last_unstale":"2023-10-30T12:58:13.006059+0000","last_undegraded":"2023-10-30T12:58:13.006059+0000","last_fullsized":"2023-10-30T12:58:13.006059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'472","last_scrub_stamp":"2023-10-30T12:58:13.005836+0000","last_deep_scrub":"53'472","last_deep_scrub_stamp":"2023-10-30T12:58:13.005836+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.005836+0000","objects_scrubbed":155,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:44:43.557965+0000","scrub_duration":0.27633308600000001,"objects_trimmed":0,"snaptrim_duration":0.0051352919999999996,"stat_sum":{"num_bytes":11222992,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":305,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"53'363","reported_seq":434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.626957+0000","last_change":"2023-10-30T12:57:44.627005+0000","last_active":"2023-10-30T12:57:44.626957+0000","last_peered":"2023-10-30T12:57:44.626957+0000","last_clean":"2023-10-30T12:57:44.626957+0000","last_became_active":"2023-10-30T12:45:18.112387+0000","last_became_peered":"2023-10-30T12:45:18.112387+0000","last_unstale":"2023-10-30T12:57:44.626957+0000","last_undegraded":"2023-10-30T12:57:44.626957+0000","last_fullsized":"2023-10-30T12:57:44.626957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'363","last_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_deep_scrub":"53'363","last_deep_scrub_stamp":"2023-10-30T12:57:44.626874+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.626874+0000","objects_scrubbed":9,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:58:54.857541+0000","scrub_duration":0.036431423999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":6934,"num_write":357,"num_write_kb":8418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31162,"num_omap_keys":58,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"53'521","reported_seq":2663,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:45.826274+0000","last_change":"2023-10-30T12:57:45.826319+0000","last_active":"2023-10-30T12:57:45.826274+0000","last_peered":"2023-10-30T12:57:45.826274+0000","last_clean":"2023-10-30T12:57:45.826274+0000","last_became_active":"2023-10-30T12:45:19.120302+0000","last_became_peered":"2023-10-30T12:45:19.120302+0000","last_unstale":"2023-10-30T12:57:45.826274+0000","last_undegraded":"2023-10-30T12:57:45.826274+0000","last_fullsized":"2023-10-30T12:57:45.826274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'521","last_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_deep_scrub":"53'521","last_deep_scrub_stamp":"2023-10-30T12:57:45.826203+0000","last_clean_scrub_stamp":"2023-10-30T12:57:45.826203+0000","objects_scrubbed":179,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:48:53.989588+0000","scrub_duration":0.225502547,"objects_trimmed":0,"snaptrim_duration":0.0034585530000000001,"stat_sum":{"num_bytes":22942522,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":36,"num_read_kb":4114,"num_write":339,"num_write_kb":30776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"53'108","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.907067+0000","last_change":"2023-10-30T12:57:56.907176+0000","last_active":"2023-10-30T12:57:56.907067+0000","last_peered":"2023-10-30T12:57:56.907067+0000","last_clean":"2023-10-30T12:57:56.907067+0000","last_became_active":"2023-10-30T12:45:18.101509+0000","last_became_peered":"2023-10-30T12:45:18.101509+0000","last_unstale":"2023-10-30T12:57:56.907067+0000","last_undegraded":"2023-10-30T12:57:56.907067+0000","last_fullsized":"2023-10-30T12:57:56.907067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'108","last_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_deep_scrub":"53'108","last_deep_scrub_stamp":"2023-10-30T12:57:56.906852+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.906852+0000","objects_scrubbed":4,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:35:54.254936+0000","scrub_duration":0.023033504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":7391,"num_write":102,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19720,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'522","reported_seq":2674,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:28.983000+0000","last_change":"2023-10-30T12:58:28.983092+0000","last_active":"2023-10-30T12:58:28.983000+0000","last_peered":"2023-10-30T12:58:28.983000+0000","last_clean":"2023-10-30T12:58:28.983000+0000","last_became_active":"2023-10-30T12:45:19.121636+0000","last_became_peered":"2023-10-30T12:45:19.121636+0000","last_unstale":"2023-10-30T12:58:28.983000+0000","last_undegraded":"2023-10-30T12:58:28.983000+0000","last_fullsized":"2023-10-30T12:58:28.983000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'522","last_scrub_stamp":"2023-10-30T12:58:28.982876+0000","last_deep_scrub":"53'522","last_deep_scrub_stamp":"2023-10-30T12:58:28.982876+0000","last_clean_scrub_stamp":"2023-10-30T12:58:28.982876+0000","objects_scrubbed":165,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:32:13.527785+0000","scrub_duration":0.29141002300000002,"objects_trimmed":0,"snaptrim_duration":0.0077634599999999998,"stat_sum":{"num_bytes":19059257,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":339,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"53'404","reported_seq":472,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.447273+0000","last_change":"2023-10-30T12:58:00.447405+0000","last_active":"2023-10-30T12:58:00.447273+0000","last_peered":"2023-10-30T12:58:00.447273+0000","last_clean":"2023-10-30T12:58:00.447273+0000","last_became_active":"2023-10-30T12:45:18.111320+0000","last_became_peered":"2023-10-30T12:45:18.111320+0000","last_unstale":"2023-10-30T12:58:00.447273+0000","last_undegraded":"2023-10-30T12:58:00.447273+0000","last_fullsized":"2023-10-30T12:58:00.447273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'404","last_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_deep_scrub":"53'404","last_deep_scrub_stamp":"2023-10-30T12:58:00.447060+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.447060+0000","objects_scrubbed":6,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:40:18.599324+0000","scrub_duration":0.056206357999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":16391,"num_write":392,"num_write_kb":16593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8414,"num_omap_keys":16,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"53'496","reported_seq":2628,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:07.739754+0000","last_change":"2023-10-30T12:58:07.739899+0000","last_active":"2023-10-30T12:58:07.739754+0000","last_peered":"2023-10-30T12:58:07.739754+0000","last_clean":"2023-10-30T12:58:07.739754+0000","last_became_active":"2023-10-30T12:45:19.086322+0000","last_became_peered":"2023-10-30T12:45:19.086322+0000","last_unstale":"2023-10-30T12:58:07.739754+0000","last_undegraded":"2023-10-30T12:58:07.739754+0000","last_fullsized":"2023-10-30T12:58:07.739754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:07.739536+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:07.739536+0000","last_clean_scrub_stamp":"2023-10-30T12:58:07.739536+0000","objects_scrubbed":167,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:41:03.366820+0000","scrub_duration":0.33077558200000001,"objects_trimmed":0,"snaptrim_duration":0.0054918830000000004,"stat_sum":{"num_bytes":33969124,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":313,"num_write_kb":33330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"53'125","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:44.181850+0000","last_change":"2023-10-30T12:57:44.181906+0000","last_active":"2023-10-30T12:57:44.181850+0000","last_peered":"2023-10-30T12:57:44.181850+0000","last_clean":"2023-10-30T12:57:44.181850+0000","last_became_active":"2023-10-30T12:45:18.074878+0000","last_became_peered":"2023-10-30T12:45:18.074878+0000","last_unstale":"2023-10-30T12:57:44.181850+0000","last_undegraded":"2023-10-30T12:57:44.181850+0000","last_fullsized":"2023-10-30T12:57:44.181850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2023-10-30T12:57:44.181771+0000","last_clean_scrub_stamp":"2023-10-30T12:57:44.181771+0000","objects_scrubbed":13,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:04:46.051796+0000","scrub_duration":0.017190094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":119,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51810,"num_omap_keys":95,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"53'531","reported_seq":2665,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.175266+0000","last_change":"2023-10-30T12:58:15.175320+0000","last_active":"2023-10-30T12:58:15.175266+0000","last_peered":"2023-10-30T12:58:15.175266+0000","last_clean":"2023-10-30T12:58:15.175266+0000","last_became_active":"2023-10-30T12:45:59.304051+0000","last_became_peered":"2023-10-30T12:45:59.304051+0000","last_unstale":"2023-10-30T12:58:15.175266+0000","last_undegraded":"2023-10-30T12:58:15.175266+0000","last_fullsized":"2023-10-30T12:58:15.175266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'531","last_scrub_stamp":"2023-10-30T12:58:15.175195+0000","last_deep_scrub":"53'531","last_deep_scrub_stamp":"2023-10-30T12:58:15.175195+0000","last_clean_scrub_stamp":"2023-10-30T12:58:15.175195+0000","objects_scrubbed":174,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:01:47.138672+0000","scrub_duration":0.194958246,"objects_trimmed":0,"snaptrim_duration":0.001312276,"stat_sum":{"num_bytes":11998188,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":32,"num_read_kb":12306,"num_write":340,"num_write_kb":20086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"53'107","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.532064+0000","last_change":"2023-10-30T12:57:48.532178+0000","last_active":"2023-10-30T12:57:48.532064+0000","last_peered":"2023-10-30T12:57:48.532064+0000","last_clean":"2023-10-30T12:57:48.532064+0000","last_became_active":"2023-10-30T12:45:18.092426+0000","last_became_peered":"2023-10-30T12:45:18.092426+0000","last_unstale":"2023-10-30T12:57:48.532064+0000","last_undegraded":"2023-10-30T12:57:48.532064+0000","last_fullsized":"2023-10-30T12:57:48.532064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'107","last_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_deep_scrub":"53'107","last_deep_scrub_stamp":"2023-10-30T12:57:48.531861+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.531861+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:43:05.413034+0000","scrub_duration":0.032263541999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":87,"num_write":179,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"53'467","reported_seq":2603,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.047339+0000","last_change":"2023-10-30T12:58:00.047393+0000","last_active":"2023-10-30T12:58:00.047339+0000","last_peered":"2023-10-30T12:58:00.047339+0000","last_clean":"2023-10-30T12:58:00.047339+0000","last_became_active":"2023-10-30T12:45:19.106180+0000","last_became_peered":"2023-10-30T12:45:19.106180+0000","last_unstale":"2023-10-30T12:58:00.047339+0000","last_undegraded":"2023-10-30T12:58:00.047339+0000","last_fullsized":"2023-10-30T12:58:00.047339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'467","last_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_deep_scrub":"53'467","last_deep_scrub_stamp":"2023-10-30T12:58:00.047244+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.047244+0000","objects_scrubbed":156,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:40:00.326328+0000","scrub_duration":0.23210165099999999,"objects_trimmed":0,"snaptrim_duration":0.0062307170000000002,"stat_sum":{"num_bytes":12072627,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":5702,"num_write":307,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'170","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:59.087103+0000","last_change":"2023-10-30T12:57:59.087197+0000","last_active":"2023-10-30T12:57:59.087103+0000","last_peered":"2023-10-30T12:57:59.087103+0000","last_clean":"2023-10-30T12:57:59.087103+0000","last_became_active":"2023-10-30T12:45:18.085323+0000","last_became_peered":"2023-10-30T12:45:18.085323+0000","last_unstale":"2023-10-30T12:57:59.087103+0000","last_undegraded":"2023-10-30T12:57:59.087103+0000","last_fullsized":"2023-10-30T12:57:59.087103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'170","last_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_deep_scrub":"53'170","last_deep_scrub_stamp":"2023-10-30T12:57:59.087009+0000","last_clean_scrub_stamp":"2023-10-30T12:57:59.087009+0000","objects_scrubbed":5,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:57:31.789859+0000","scrub_duration":0.011509221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":165,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32668,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'502","reported_seq":2637,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.084264+0000","last_change":"2023-10-30T12:58:09.084320+0000","last_active":"2023-10-30T12:58:09.084264+0000","last_peered":"2023-10-30T12:58:09.084264+0000","last_clean":"2023-10-30T12:58:09.084264+0000","last_became_active":"2023-10-30T12:45:19.107827+0000","last_became_peered":"2023-10-30T12:45:19.107827+0000","last_unstale":"2023-10-30T12:58:09.084264+0000","last_undegraded":"2023-10-30T12:58:09.084264+0000","last_fullsized":"2023-10-30T12:58:09.084264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'502","last_scrub_stamp":"2023-10-30T12:58:09.084184+0000","last_deep_scrub":"53'502","last_deep_scrub_stamp":"2023-10-30T12:58:09.084184+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.084184+0000","objects_scrubbed":163,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:23:56.040279+0000","scrub_duration":0.29045587899999997,"objects_trimmed":0,"snaptrim_duration":0.0069379849999999998,"stat_sum":{"num_bytes":27073673,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":30,"num_read_kb":4114,"num_write":317,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"53'127","reported_seq":187,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:08.170192+0000","last_change":"2023-10-30T12:58:08.170243+0000","last_active":"2023-10-30T12:58:08.170192+0000","last_peered":"2023-10-30T12:58:08.170192+0000","last_clean":"2023-10-30T12:58:08.170192+0000","last_became_active":"2023-10-30T12:45:18.100716+0000","last_became_peered":"2023-10-30T12:45:18.100716+0000","last_unstale":"2023-10-30T12:58:08.170192+0000","last_undegraded":"2023-10-30T12:58:08.170192+0000","last_fullsized":"2023-10-30T12:58:08.170192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'127","last_scrub_stamp":"2023-10-30T12:58:08.170113+0000","last_deep_scrub":"53'127","last_deep_scrub_stamp":"2023-10-30T12:58:08.170113+0000","last_clean_scrub_stamp":"2023-10-30T12:58:08.170113+0000","objects_scrubbed":4,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:16:44.348618+0000","scrub_duration":0.016712257000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20202,"num_omap_keys":37,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"53'561","reported_seq":2697,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.335935+0000","last_change":"2023-10-30T12:58:13.336055+0000","last_active":"2023-10-30T12:58:13.335935+0000","last_peered":"2023-10-30T12:58:13.335935+0000","last_clean":"2023-10-30T12:58:13.335935+0000","last_became_active":"2023-10-30T12:45:19.123562+0000","last_became_peered":"2023-10-30T12:45:19.123562+0000","last_unstale":"2023-10-30T12:58:13.335935+0000","last_undegraded":"2023-10-30T12:58:13.335935+0000","last_fullsized":"2023-10-30T12:58:13.335935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'561","last_scrub_stamp":"2023-10-30T12:58:13.335723+0000","last_deep_scrub":"53'561","last_deep_scrub_stamp":"2023-10-30T12:58:13.335723+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.335723+0000","objects_scrubbed":194,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:21:26.649100+0000","scrub_duration":0.32796992200000002,"objects_trimmed":0,"snaptrim_duration":0.0068258499999999996,"stat_sum":{"num_bytes":14526751,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":358,"num_write_kb":14373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"53'19","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.062670+0000","last_change":"2023-10-30T12:58:09.062763+0000","last_active":"2023-10-30T12:58:09.062670+0000","last_peered":"2023-10-30T12:58:09.062670+0000","last_clean":"2023-10-30T12:58:09.062670+0000","last_became_active":"2023-10-30T12:45:18.098997+0000","last_became_peered":"2023-10-30T12:45:18.098997+0000","last_unstale":"2023-10-30T12:58:09.062670+0000","last_undegraded":"2023-10-30T12:58:09.062670+0000","last_fullsized":"2023-10-30T12:58:09.062670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2023-10-30T12:58:09.062573+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2023-10-30T12:58:09.062573+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.062573+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:11:55.643732+0000","scrub_duration":0.0070198639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":16,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30824,"num_omap_keys":57,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"53'496","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:17.968850+0000","last_change":"2023-10-30T12:58:17.968907+0000","last_active":"2023-10-30T12:58:17.968850+0000","last_peered":"2023-10-30T12:58:17.968850+0000","last_clean":"2023-10-30T12:58:17.968850+0000","last_became_active":"2023-10-30T12:45:19.126449+0000","last_became_peered":"2023-10-30T12:45:19.126449+0000","last_unstale":"2023-10-30T12:58:17.968850+0000","last_undegraded":"2023-10-30T12:58:17.968850+0000","last_fullsized":"2023-10-30T12:58:17.968850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-30T12:58:17.968753+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-30T12:58:17.968753+0000","last_clean_scrub_stamp":"2023-10-30T12:58:17.968753+0000","objects_scrubbed":163,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:15:36.268957+0000","scrub_duration":0.23173764799999999,"objects_trimmed":0,"snaptrim_duration":0.0073547990000000004,"stat_sum":{"num_bytes":11468449,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":28,"num_read_kb":14,"num_write":323,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.811840+0000","last_change":"2023-10-30T12:57:48.811953+0000","last_active":"2023-10-30T12:57:48.811840+0000","last_peered":"2023-10-30T12:57:48.811840+0000","last_clean":"2023-10-30T12:57:48.811840+0000","last_became_active":"2023-10-30T12:45:07.074092+0000","last_became_peered":"2023-10-30T12:45:07.074092+0000","last_unstale":"2023-10-30T12:57:48.811840+0000","last_undegraded":"2023-10-30T12:57:48.811840+0000","last_fullsized":"2023-10-30T12:57:48.811840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-30T12:57:48.811641+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.811641+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:49:27.857700+0000","scrub_duration":0.0086407700000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"53'15","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.871134+0000","last_change":"2023-10-30T12:57:49.871261+0000","last_active":"2023-10-30T12:57:49.871134+0000","last_peered":"2023-10-30T12:57:49.871134+0000","last_clean":"2023-10-30T12:57:49.871134+0000","last_became_active":"2023-10-30T12:45:18.102976+0000","last_became_peered":"2023-10-30T12:45:18.102976+0000","last_unstale":"2023-10-30T12:57:49.871134+0000","last_undegraded":"2023-10-30T12:57:49.871134+0000","last_fullsized":"2023-10-30T12:57:49.871134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:49.870906+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.870906+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:10:34.313354+0000","scrub_duration":0.010713916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":13,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15500,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'456","reported_seq":2587,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:21.594862+0000","last_change":"2023-10-30T12:58:21.594938+0000","last_active":"2023-10-30T12:58:21.594862+0000","last_peered":"2023-10-30T12:58:21.594862+0000","last_clean":"2023-10-30T12:58:21.594862+0000","last_became_active":"2023-10-30T12:45:59.539405+0000","last_became_peered":"2023-10-30T12:45:59.539405+0000","last_unstale":"2023-10-30T12:58:21.594862+0000","last_undegraded":"2023-10-30T12:58:21.594862+0000","last_fullsized":"2023-10-30T12:58:21.594862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'456","last_scrub_stamp":"2023-10-30T12:58:21.594787+0000","last_deep_scrub":"53'456","last_deep_scrub_stamp":"2023-10-30T12:58:21.594787+0000","last_clean_scrub_stamp":"2023-10-30T12:58:21.594787+0000","objects_scrubbed":146,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:52:49.424663+0000","scrub_duration":0.12018036999999999,"objects_trimmed":0,"snaptrim_duration":0.0028873000000000002,"stat_sum":{"num_bytes":14536769,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":295,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"53'505","reported_seq":574,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:50.786677+0000","last_change":"2023-10-30T12:57:50.786811+0000","last_active":"2023-10-30T12:57:50.786677+0000","last_peered":"2023-10-30T12:57:50.786677+0000","last_clean":"2023-10-30T12:57:50.786677+0000","last_became_active":"2023-10-30T12:45:18.117007+0000","last_became_peered":"2023-10-30T12:45:18.117007+0000","last_unstale":"2023-10-30T12:57:50.786677+0000","last_undegraded":"2023-10-30T12:57:50.786677+0000","last_fullsized":"2023-10-30T12:57:50.786677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-30T12:57:50.786437+0000","last_clean_scrub_stamp":"2023-10-30T12:57:50.786437+0000","objects_scrubbed":8,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:11:51.266608+0000","scrub_duration":0.043680202000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":16399,"num_write":498,"num_write_kb":12661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":44444,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'489","reported_seq":2612,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:06.635929+0000","last_change":"2023-10-30T12:58:06.635989+0000","last_active":"2023-10-30T12:58:06.635929+0000","last_peered":"2023-10-30T12:58:06.635929+0000","last_clean":"2023-10-30T12:58:06.635929+0000","last_became_active":"2023-10-30T12:45:19.121198+0000","last_became_peered":"2023-10-30T12:45:19.121198+0000","last_unstale":"2023-10-30T12:58:06.635929+0000","last_undegraded":"2023-10-30T12:58:06.635929+0000","last_fullsized":"2023-10-30T12:58:06.635929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'489","last_scrub_stamp":"2023-10-30T12:58:06.635853+0000","last_deep_scrub":"53'489","last_deep_scrub_stamp":"2023-10-30T12:58:06.635853+0000","last_clean_scrub_stamp":"2023-10-30T12:58:06.635853+0000","objects_scrubbed":162,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:55:34.992256+0000","scrub_duration":0.25796478499999997,"objects_trimmed":0,"snaptrim_duration":0.0024928070000000001,"stat_sum":{"num_bytes":14741040,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":307,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"53'164","reported_seq":289,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:39.887849+0000","last_change":"2023-10-30T12:57:47.115495+0000","last_active":"2023-10-30T12:58:39.887849+0000","last_peered":"2023-10-30T12:58:39.887849+0000","last_clean":"2023-10-30T12:58:39.887849+0000","last_became_active":"2023-10-30T12:45:18.085632+0000","last_became_peered":"2023-10-30T12:45:18.085632+0000","last_unstale":"2023-10-30T12:58:39.887849+0000","last_undegraded":"2023-10-30T12:58:39.887849+0000","last_fullsized":"2023-10-30T12:58:39.887849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:47.115321+0000","last_clean_scrub_stamp":"2023-10-30T12:57:47.115321+0000","objects_scrubbed":9,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:48:57.832139+0000","scrub_duration":0.015406534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4557088,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":75,"num_read_kb":4257,"num_write":153,"num_write_kb":4609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63882,"num_omap_keys":117,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'493","reported_seq":2635,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:11.291688+0000","last_change":"2023-10-30T12:58:11.291745+0000","last_active":"2023-10-30T12:58:11.291688+0000","last_peered":"2023-10-30T12:58:11.291688+0000","last_clean":"2023-10-30T12:58:11.291688+0000","last_became_active":"2023-10-30T12:45:19.075267+0000","last_became_peered":"2023-10-30T12:45:19.075267+0000","last_unstale":"2023-10-30T12:58:11.291688+0000","last_undegraded":"2023-10-30T12:58:11.291688+0000","last_fullsized":"2023-10-30T12:58:11.291688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-30T12:58:11.291608+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-30T12:58:11.291608+0000","last_clean_scrub_stamp":"2023-10-30T12:58:11.291608+0000","objects_scrubbed":168,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:34:58.191035+0000","scrub_duration":0.14191978399999999,"objects_trimmed":0,"snaptrim_duration":0.002543221,"stat_sum":{"num_bytes":18173135,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":36,"num_read_kb":21,"num_write":309,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"53'249","reported_seq":311,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.994818+0000","last_change":"2023-10-30T12:58:00.994936+0000","last_active":"2023-10-30T12:58:00.994818+0000","last_peered":"2023-10-30T12:58:00.994818+0000","last_clean":"2023-10-30T12:58:00.994818+0000","last_became_active":"2023-10-30T12:45:18.103888+0000","last_became_peered":"2023-10-30T12:45:18.103888+0000","last_unstale":"2023-10-30T12:58:00.994818+0000","last_undegraded":"2023-10-30T12:58:00.994818+0000","last_fullsized":"2023-10-30T12:58:00.994818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'249","last_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_deep_scrub":"53'249","last_deep_scrub_stamp":"2023-10-30T12:58:00.994611+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.994611+0000","objects_scrubbed":4,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:51:45.189479+0000","scrub_duration":0.037553993000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":240,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21832,"num_omap_keys":40,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'541","reported_seq":2671,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:52.766867+0000","last_change":"2023-10-30T12:57:52.767026+0000","last_active":"2023-10-30T12:57:52.766867+0000","last_peered":"2023-10-30T12:57:52.766867+0000","last_clean":"2023-10-30T12:57:52.766867+0000","last_became_active":"2023-10-30T12:45:19.118617+0000","last_became_peered":"2023-10-30T12:45:19.118617+0000","last_unstale":"2023-10-30T12:57:52.766867+0000","last_undegraded":"2023-10-30T12:57:52.766867+0000","last_fullsized":"2023-10-30T12:57:52.766867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'541","last_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_deep_scrub":"53'541","last_deep_scrub_stamp":"2023-10-30T12:57:52.766665+0000","last_clean_scrub_stamp":"2023-10-30T12:57:52.766665+0000","objects_scrubbed":172,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:47:06.068606+0000","scrub_duration":0.32093353800000002,"objects_trimmed":0,"snaptrim_duration":0.0021853770000000001,"stat_sum":{"num_bytes":14926864,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":24,"num_read_kb":12,"num_write":354,"num_write_kb":14766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"53'4","reported_seq":64,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:09.124470+0000","last_change":"2023-10-30T12:58:09.124515+0000","last_active":"2023-10-30T12:58:09.124470+0000","last_peered":"2023-10-30T12:58:09.124470+0000","last_clean":"2023-10-30T12:58:09.124470+0000","last_became_active":"2023-10-30T12:45:18.081306+0000","last_became_peered":"2023-10-30T12:45:18.081306+0000","last_unstale":"2023-10-30T12:58:09.124470+0000","last_undegraded":"2023-10-30T12:58:09.124470+0000","last_fullsized":"2023-10-30T12:58:09.124470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2023-10-30T12:58:09.124398+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2023-10-30T12:58:09.124398+0000","last_clean_scrub_stamp":"2023-10-30T12:58:09.124398+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:50:41.929762+0000","scrub_duration":0.003007633,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13058,"num_omap_keys":24,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'495","reported_seq":2624,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:39.361060+0000","last_change":"2023-10-30T12:58:39.361297+0000","last_active":"2023-10-30T12:58:39.361060+0000","last_peered":"2023-10-30T12:58:39.361060+0000","last_clean":"2023-10-30T12:58:39.361060+0000","last_became_active":"2023-10-30T12:45:19.119854+0000","last_became_peered":"2023-10-30T12:45:19.119854+0000","last_unstale":"2023-10-30T12:58:39.361060+0000","last_undegraded":"2023-10-30T12:58:39.361060+0000","last_fullsized":"2023-10-30T12:58:39.361060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:58:39.360806+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:58:39.360806+0000","last_clean_scrub_stamp":"2023-10-30T12:58:39.360806+0000","objects_scrubbed":165,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:58:35.819351+0000","scrub_duration":0.30520466299999999,"objects_trimmed":0,"snaptrim_duration":0.0074281679999999998,"stat_sum":{"num_bytes":9698880,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":23,"num_read_kb":13,"num_write":313,"num_write_kb":13729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"53'15","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:58.813043+0000","last_change":"2023-10-30T12:57:58.813159+0000","last_active":"2023-10-30T12:57:58.813043+0000","last_peered":"2023-10-30T12:57:58.813043+0000","last_clean":"2023-10-30T12:57:58.813043+0000","last_became_active":"2023-10-30T12:45:18.119807+0000","last_became_peered":"2023-10-30T12:45:18.119807+0000","last_unstale":"2023-10-30T12:57:58.813043+0000","last_undegraded":"2023-10-30T12:57:58.813043+0000","last_fullsized":"2023-10-30T12:57:58.813043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2023-10-30T12:57:58.812837+0000","last_clean_scrub_stamp":"2023-10-30T12:57:58.812837+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:00:16.476751+0000","scrub_duration":0.009639435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25372,"num_omap_keys":48,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'514","reported_seq":2650,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:22.292021+0000","last_change":"2023-10-30T12:58:22.292097+0000","last_active":"2023-10-30T12:58:22.292021+0000","last_peered":"2023-10-30T12:58:22.292021+0000","last_clean":"2023-10-30T12:58:22.292021+0000","last_became_active":"2023-10-30T12:45:19.077884+0000","last_became_peered":"2023-10-30T12:45:19.077884+0000","last_unstale":"2023-10-30T12:58:22.292021+0000","last_undegraded":"2023-10-30T12:58:22.292021+0000","last_fullsized":"2023-10-30T12:58:22.292021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'514","last_scrub_stamp":"2023-10-30T12:58:22.291946+0000","last_deep_scrub":"53'514","last_deep_scrub_stamp":"2023-10-30T12:58:22.291946+0000","last_clean_scrub_stamp":"2023-10-30T12:58:22.291946+0000","objects_scrubbed":173,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:12:03.363935+0000","scrub_duration":0.133602522,"objects_trimmed":0,"snaptrim_duration":0.0026219160000000002,"stat_sum":{"num_bytes":5814763,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":30,"num_read_kb":18,"num_write":327,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"53'102","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.157983+0000","last_change":"2023-10-30T12:58:10.157983+0000","last_active":"2023-10-30T12:58:10.157983+0000","last_peered":"2023-10-30T12:58:10.157983+0000","last_clean":"2023-10-30T12:58:10.157983+0000","last_became_active":"2023-10-30T12:45:18.084751+0000","last_became_peered":"2023-10-30T12:45:18.084751+0000","last_unstale":"2023-10-30T12:58:10.157983+0000","last_undegraded":"2023-10-30T12:58:10.157983+0000","last_fullsized":"2023-10-30T12:58:10.157983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2023-10-30T12:58:10.157787+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2023-10-30T12:58:10.157787+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.157787+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:54:54.361702+0000","scrub_duration":0.053518498999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":240,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11565,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'451","reported_seq":2580,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:08.583430+0000","last_change":"2023-10-30T12:58:08.583430+0000","last_active":"2023-10-30T12:58:08.583430+0000","last_peered":"2023-10-30T12:58:08.583430+0000","last_clean":"2023-10-30T12:58:08.583430+0000","last_became_active":"2023-10-30T12:45:19.112337+0000","last_became_peered":"2023-10-30T12:45:19.112337+0000","last_unstale":"2023-10-30T12:58:08.583430+0000","last_undegraded":"2023-10-30T12:58:08.583430+0000","last_fullsized":"2023-10-30T12:58:08.583430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'451","last_scrub_stamp":"2023-10-30T12:58:08.583306+0000","last_deep_scrub":"53'451","last_deep_scrub_stamp":"2023-10-30T12:58:08.583306+0000","last_clean_scrub_stamp":"2023-10-30T12:58:08.583306+0000","objects_scrubbed":138,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:46:20.024190+0000","scrub_duration":0.12646469299999999,"objects_trimmed":0,"snaptrim_duration":0.0029345270000000001,"stat_sum":{"num_bytes":5482671,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":298,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"53'16","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:51.452812+0000","last_change":"2023-10-30T12:57:51.452812+0000","last_active":"2023-10-30T12:57:51.452812+0000","last_peered":"2023-10-30T12:57:51.452812+0000","last_clean":"2023-10-30T12:57:51.452812+0000","last_became_active":"2023-10-30T12:45:18.117303+0000","last_became_peered":"2023-10-30T12:45:18.117303+0000","last_unstale":"2023-10-30T12:57:51.452812+0000","last_undegraded":"2023-10-30T12:57:51.452812+0000","last_fullsized":"2023-10-30T12:57:51.452812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-30T12:57:51.452333+0000","last_clean_scrub_stamp":"2023-10-30T12:57:51.452333+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:18:15.321071+0000","scrub_duration":0.011134068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4360,"num_omap_keys":8,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'484","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:48.782635+0000","last_change":"2023-10-30T12:57:48.782635+0000","last_active":"2023-10-30T12:57:48.782635+0000","last_peered":"2023-10-30T12:57:48.782635+0000","last_clean":"2023-10-30T12:57:48.782635+0000","last_became_active":"2023-10-30T12:45:19.089686+0000","last_became_peered":"2023-10-30T12:45:19.089686+0000","last_unstale":"2023-10-30T12:57:48.782635+0000","last_undegraded":"2023-10-30T12:57:48.782635+0000","last_fullsized":"2023-10-30T12:57:48.782635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-30T12:57:48.782504+0000","last_clean_scrub_stamp":"2023-10-30T12:57:48.782504+0000","objects_scrubbed":167,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:31:48.560996+0000","scrub_duration":0.129876255,"objects_trimmed":0,"snaptrim_duration":0.0032574219999999998,"stat_sum":{"num_bytes":10732563,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":315,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"53'167","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:49.808170+0000","last_change":"2023-10-30T12:57:49.808170+0000","last_active":"2023-10-30T12:57:49.808170+0000","last_peered":"2023-10-30T12:57:49.808170+0000","last_clean":"2023-10-30T12:57:49.808170+0000","last_became_active":"2023-10-30T12:45:18.116920+0000","last_became_peered":"2023-10-30T12:45:18.116920+0000","last_unstale":"2023-10-30T12:57:49.808170+0000","last_undegraded":"2023-10-30T12:57:49.808170+0000","last_fullsized":"2023-10-30T12:57:49.808170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'167","last_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_deep_scrub":"53'167","last_deep_scrub_stamp":"2023-10-30T12:57:49.807844+0000","last_clean_scrub_stamp":"2023-10-30T12:57:49.807844+0000","objects_scrubbed":4,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:04:19.047233+0000","scrub_duration":0.025125933999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":160,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6708,"num_omap_keys":13,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"53'495","reported_seq":2620,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:54.741397+0000","last_change":"2023-10-30T12:57:54.741397+0000","last_active":"2023-10-30T12:57:54.741397+0000","last_peered":"2023-10-30T12:57:54.741397+0000","last_clean":"2023-10-30T12:57:54.741397+0000","last_became_active":"2023-10-30T12:45:19.091879+0000","last_became_peered":"2023-10-30T12:45:19.091879+0000","last_unstale":"2023-10-30T12:57:54.741397+0000","last_undegraded":"2023-10-30T12:57:54.741397+0000","last_fullsized":"2023-10-30T12:57:54.741397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-30T12:57:54.741074+0000","last_clean_scrub_stamp":"2023-10-30T12:57:54.741074+0000","objects_scrubbed":156,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:21:22.048659+0000","scrub_duration":0.30326803000000002,"objects_trimmed":0,"snaptrim_duration":0.0027700799999999999,"stat_sum":{"num_bytes":10418507,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":19,"num_read_kb":7,"num_write":325,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"53'67","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:06.182656+0000","last_change":"2023-10-30T12:58:06.182656+0000","last_active":"2023-10-30T12:58:06.182656+0000","last_peered":"2023-10-30T12:58:06.182656+0000","last_clean":"2023-10-30T12:58:06.182656+0000","last_became_active":"2023-10-30T12:45:18.113614+0000","last_became_peered":"2023-10-30T12:45:18.113614+0000","last_unstale":"2023-10-30T12:58:06.182656+0000","last_undegraded":"2023-10-30T12:58:06.182656+0000","last_fullsized":"2023-10-30T12:58:06.182656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'67","last_scrub_stamp":"2023-10-30T12:58:06.182512+0000","last_deep_scrub":"53'67","last_deep_scrub_stamp":"2023-10-30T12:58:06.182512+0000","last_clean_scrub_stamp":"2023-10-30T12:58:06.182512+0000","objects_scrubbed":9,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T20:52:30.629159+0000","scrub_duration":0.023539636999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":15,"num_read_kb":4110,"num_write":64,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":61026,"num_omap_keys":114,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'487","reported_seq":2625,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:13.178881+0000","last_change":"2023-10-30T12:58:13.178938+0000","last_active":"2023-10-30T12:58:13.178881+0000","last_peered":"2023-10-30T12:58:13.178881+0000","last_clean":"2023-10-30T12:58:13.178881+0000","last_became_active":"2023-10-30T12:45:59.538198+0000","last_became_peered":"2023-10-30T12:45:59.538198+0000","last_unstale":"2023-10-30T12:58:13.178881+0000","last_undegraded":"2023-10-30T12:58:13.178881+0000","last_fullsized":"2023-10-30T12:58:13.178881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:13.178803+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:13.178803+0000","last_clean_scrub_stamp":"2023-10-30T12:58:13.178803+0000","objects_scrubbed":160,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:04:19.440362+0000","scrub_duration":0.131130357,"objects_trimmed":0,"snaptrim_duration":0.001640163,"stat_sum":{"num_bytes":5889364,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":316,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"53'122","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:00.127400+0000","last_change":"2023-10-30T12:58:00.127400+0000","last_active":"2023-10-30T12:58:00.127400+0000","last_peered":"2023-10-30T12:58:00.127400+0000","last_clean":"2023-10-30T12:58:00.127400+0000","last_became_active":"2023-10-30T12:45:18.083575+0000","last_became_peered":"2023-10-30T12:45:18.083575+0000","last_unstale":"2023-10-30T12:58:00.127400+0000","last_undegraded":"2023-10-30T12:58:00.127400+0000","last_fullsized":"2023-10-30T12:58:00.127400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'122","last_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_deep_scrub":"53'122","last_deep_scrub_stamp":"2023-10-30T12:58:00.127231+0000","last_clean_scrub_stamp":"2023-10-30T12:58:00.127231+0000","objects_scrubbed":4,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:06:56.254642+0000","scrub_duration":0.026565129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":33,"num_write":199,"num_write_kb":4799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18104,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"53'486","reported_seq":2614,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:19.540348+0000","last_change":"2023-10-30T12:58:19.540393+0000","last_active":"2023-10-30T12:58:19.540348+0000","last_peered":"2023-10-30T12:58:19.540348+0000","last_clean":"2023-10-30T12:58:19.540348+0000","last_became_active":"2023-10-30T12:45:19.097564+0000","last_became_peered":"2023-10-30T12:45:19.097564+0000","last_unstale":"2023-10-30T12:58:19.540348+0000","last_undegraded":"2023-10-30T12:58:19.540348+0000","last_fullsized":"2023-10-30T12:58:19.540348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'486","last_scrub_stamp":"2023-10-30T12:58:19.540267+0000","last_deep_scrub":"53'486","last_deep_scrub_stamp":"2023-10-30T12:58:19.540267+0000","last_clean_scrub_stamp":"2023-10-30T12:58:19.540267+0000","objects_scrubbed":156,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:44:39.118000+0000","scrub_duration":0.122920026,"objects_trimmed":0,"snaptrim_duration":0.0023431749999999999,"stat_sum":{"num_bytes":10753512,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":23,"num_read_kb":16,"num_write":319,"num_write_kb":10640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"53'192","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:56.821533+0000","last_change":"2023-10-30T12:57:56.821734+0000","last_active":"2023-10-30T12:57:56.821533+0000","last_peered":"2023-10-30T12:57:56.821533+0000","last_clean":"2023-10-30T12:57:56.821533+0000","last_became_active":"2023-10-30T12:45:18.120452+0000","last_became_peered":"2023-10-30T12:45:18.120452+0000","last_unstale":"2023-10-30T12:57:56.821533+0000","last_undegraded":"2023-10-30T12:57:56.821533+0000","last_fullsized":"2023-10-30T12:57:56.821533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'192","last_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_deep_scrub":"53'192","last_deep_scrub_stamp":"2023-10-30T12:57:56.821278+0000","last_clean_scrub_stamp":"2023-10-30T12:57:56.821278+0000","objects_scrubbed":5,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:54:26.335029+0000","scrub_duration":0.022744626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":4107,"num_write":183,"num_write_kb":4257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"53'488","reported_seq":2623,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:01.602376+0000","last_change":"2023-10-30T12:58:01.602376+0000","last_active":"2023-10-30T12:58:01.602376+0000","last_peered":"2023-10-30T12:58:01.602376+0000","last_clean":"2023-10-30T12:58:01.602376+0000","last_became_active":"2023-10-30T12:45:19.094183+0000","last_became_peered":"2023-10-30T12:45:19.094183+0000","last_unstale":"2023-10-30T12:58:01.602376+0000","last_undegraded":"2023-10-30T12:58:01.602376+0000","last_fullsized":"2023-10-30T12:58:01.602376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'488","last_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_deep_scrub":"53'488","last_deep_scrub_stamp":"2023-10-30T12:58:01.602238+0000","last_clean_scrub_stamp":"2023-10-30T12:58:01.602238+0000","objects_scrubbed":157,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:32:01.459702+0000","scrub_duration":0.22063908099999999,"objects_trimmed":0,"snaptrim_duration":0.0029911949999999999,"stat_sum":{"num_bytes":14613235,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":316,"num_write_kb":18469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"53'483","reported_seq":2623,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:42.300407+0000","last_change":"2023-10-30T12:58:42.300497+0000","last_active":"2023-10-30T12:58:42.300407+0000","last_peered":"2023-10-30T12:58:42.300407+0000","last_clean":"2023-10-30T12:58:42.300407+0000","last_became_active":"2023-10-30T12:45:19.130274+0000","last_became_peered":"2023-10-30T12:45:19.130274+0000","last_unstale":"2023-10-30T12:58:42.300407+0000","last_undegraded":"2023-10-30T12:58:42.300407+0000","last_fullsized":"2023-10-30T12:58:42.300407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'483","last_scrub_stamp":"2023-10-30T12:58:42.300290+0000","last_deep_scrub":"53'483","last_deep_scrub_stamp":"2023-10-30T12:58:42.300290+0000","last_clean_scrub_stamp":"2023-10-30T12:58:42.300290+0000","objects_scrubbed":149,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:32.391227+0000","scrub_duration":0.27418667099999999,"objects_trimmed":0,"snaptrim_duration":0.0073504720000000003,"stat_sum":{"num_bytes":22516204,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":313,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"53'154","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.745224+0000","last_change":"2023-10-30T12:57:53.745301+0000","last_active":"2023-10-30T12:57:53.745224+0000","last_peered":"2023-10-30T12:57:53.745224+0000","last_clean":"2023-10-30T12:57:53.745224+0000","last_became_active":"2023-10-30T12:45:18.116792+0000","last_became_peered":"2023-10-30T12:45:18.116792+0000","last_unstale":"2023-10-30T12:57:53.745224+0000","last_undegraded":"2023-10-30T12:57:53.745224+0000","last_fullsized":"2023-10-30T12:57:53.745224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'154","last_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_deep_scrub":"53'154","last_deep_scrub_stamp":"2023-10-30T12:57:53.745078+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.745078+0000","objects_scrubbed":3,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T19:28:27.394467+0000","scrub_duration":0.016168252000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":148,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"53'487","reported_seq":2625,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:15.211256+0000","last_change":"2023-10-30T12:58:15.211299+0000","last_active":"2023-10-30T12:58:15.211256+0000","last_peered":"2023-10-30T12:58:15.211256+0000","last_clean":"2023-10-30T12:58:15.211256+0000","last_became_active":"2023-10-30T12:45:19.131056+0000","last_became_peered":"2023-10-30T12:45:19.131056+0000","last_unstale":"2023-10-30T12:58:15.211256+0000","last_undegraded":"2023-10-30T12:58:15.211256+0000","last_fullsized":"2023-10-30T12:58:15.211256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'487","last_scrub_stamp":"2023-10-30T12:58:15.211165+0000","last_deep_scrub":"53'487","last_deep_scrub_stamp":"2023-10-30T12:58:15.211165+0000","last_clean_scrub_stamp":"2023-10-30T12:58:15.211165+0000","objects_scrubbed":152,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:05:41.398922+0000","scrub_duration":0.18129031300000001,"objects_trimmed":0,"snaptrim_duration":0.0070141020000000004,"stat_sum":{"num_bytes":10242846,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":314,"num_write_kb":14255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"53'369","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:55.867694+0000","last_change":"2023-10-30T12:57:55.867838+0000","last_active":"2023-10-30T12:57:55.867694+0000","last_peered":"2023-10-30T12:57:55.867694+0000","last_clean":"2023-10-30T12:57:55.867694+0000","last_became_active":"2023-10-30T12:45:18.116843+0000","last_became_peered":"2023-10-30T12:45:18.116843+0000","last_unstale":"2023-10-30T12:57:55.867694+0000","last_undegraded":"2023-10-30T12:57:55.867694+0000","last_fullsized":"2023-10-30T12:57:55.867694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'369","last_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_deep_scrub":"53'369","last_deep_scrub_stamp":"2023-10-30T12:57:55.867484+0000","last_clean_scrub_stamp":"2023-10-30T12:57:55.867484+0000","objects_scrubbed":7,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T21:44:44.299714+0000","scrub_duration":0.065173216000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":11133,"num_write":356,"num_write_kb":16540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":22,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"53'482","reported_seq":2610,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:14.587594+0000","last_change":"2023-10-30T12:58:14.587674+0000","last_active":"2023-10-30T12:58:14.587594+0000","last_peered":"2023-10-30T12:58:14.587594+0000","last_clean":"2023-10-30T12:58:14.587594+0000","last_became_active":"2023-10-30T12:45:19.130951+0000","last_became_peered":"2023-10-30T12:45:19.130951+0000","last_unstale":"2023-10-30T12:58:14.587594+0000","last_undegraded":"2023-10-30T12:58:14.587594+0000","last_fullsized":"2023-10-30T12:58:14.587594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'482","last_scrub_stamp":"2023-10-30T12:58:14.587519+0000","last_deep_scrub":"53'482","last_deep_scrub_stamp":"2023-10-30T12:58:14.587519+0000","last_clean_scrub_stamp":"2023-10-30T12:58:14.587519+0000","objects_scrubbed":147,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:39:51.524443+0000","scrub_duration":0.129591384,"objects_trimmed":0,"snaptrim_duration":0.0035966930000000002,"stat_sum":{"num_bytes":7604824,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":23,"num_read_kb":17,"num_write":321,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"53'281","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:53.503383+0000","last_change":"2023-10-30T12:57:53.503441+0000","last_active":"2023-10-30T12:57:53.503383+0000","last_peered":"2023-10-30T12:57:53.503383+0000","last_clean":"2023-10-30T12:57:53.503383+0000","last_became_active":"2023-10-30T12:45:18.089937+0000","last_became_peered":"2023-10-30T12:45:18.089937+0000","last_unstale":"2023-10-30T12:57:53.503383+0000","last_undegraded":"2023-10-30T12:57:53.503383+0000","last_fullsized":"2023-10-30T12:57:53.503383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'281","last_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_deep_scrub":"53'281","last_deep_scrub_stamp":"2023-10-30T12:57:53.503283+0000","last_clean_scrub_stamp":"2023-10-30T12:57:53.503283+0000","objects_scrubbed":3,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-01T00:35:22.524008+0000","scrub_duration":0.021426580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":9176,"num_write":273,"num_write_kb":8344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8098,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"53'489","reported_seq":2617,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:58:10.723690+0000","last_change":"2023-10-30T12:58:10.723733+0000","last_active":"2023-10-30T12:58:10.723690+0000","last_peered":"2023-10-30T12:58:10.723690+0000","last_clean":"2023-10-30T12:58:10.723690+0000","last_became_active":"2023-10-30T12:45:19.127613+0000","last_became_peered":"2023-10-30T12:45:19.127613+0000","last_unstale":"2023-10-30T12:58:10.723690+0000","last_undegraded":"2023-10-30T12:58:10.723690+0000","last_fullsized":"2023-10-30T12:58:10.723690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'489","last_scrub_stamp":"2023-10-30T12:58:10.723608+0000","last_deep_scrub":"53'489","last_deep_scrub_stamp":"2023-10-30T12:58:10.723608+0000","last_clean_scrub_stamp":"2023-10-30T12:58:10.723608+0000","objects_scrubbed":158,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T23:38:16.558220+0000","scrub_duration":0.19531570600000001,"objects_trimmed":0,"snaptrim_duration":0.0025871240000000001,"stat_sum":{"num_bytes":21920059,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":22,"num_read_kb":11,"num_write":318,"num_write_kb":21570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"53'201","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T12:57:43.639282+0000","last_change":"2023-10-30T12:57:43.639282+0000","last_active":"2023-10-30T12:57:43.639282+0000","last_peered":"2023-10-30T12:57:43.639282+0000","last_clean":"2023-10-30T12:57:43.639282+0000","last_became_active":"2023-10-30T12:45:18.119554+0000","last_became_peered":"2023-10-30T12:45:18.119554+0000","last_unstale":"2023-10-30T12:57:43.639282+0000","last_undegraded":"2023-10-30T12:57:43.639282+0000","last_fullsized":"2023-10-30T12:57:43.639282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'201","last_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_deep_scrub":"53'201","last_deep_scrub_stamp":"2023-10-30T12:57:43.639154+0000","last_clean_scrub_stamp":"2023-10-30T12:57:43.639154+0000","objects_scrubbed":8,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T22:23:04.340919+0000","scrub_duration":0.033784741,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":8226,"num_write":195,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9828,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":925184388,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":1831,"num_read_kb":100183,"num_write":20887,"num_write_kb":1047691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1902518272,"data_stored":1848828680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32123,"ondisk_log_size":32123,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":177869210,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":601,"num_read_kb":143265,"num_write":5714,"num_write_kb":179487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":103,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":692939,"num_omap_keys":1293,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":355901440,"data_stored":355738420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5609,"ondisk_log_size":5609,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403156,"kb_used_data":327564,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":93968684,"statfs":{"total":96636764160,"available":96223932416,"internally_reserved":0,"allocated":335425536,"data_stored":327567493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77396427},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.097,"15min":0.89700000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.436},"max":{"1min":1.0800000000000001,"5min":26.132999999999999,"15min":26.132999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":1.302,"15min":1.0389999999999999},"min":{"1min":0.63300000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2110000000000001,"5min":25.327000000000002,"15min":25.327000000000002},"last":1.0369999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.119,"15min":0.93600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7150000000000001,"5min":26.803999999999998,"15min":26.803999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.2809999999999999,"15min":1.056},"min":{"1min":0.68500000000000005,"5min":0.53800000000000003,"15min":0.53300000000000003},"max":{"1min":1.8480000000000001,"5min":26.838000000000001,"15min":26.838000000000001},"last":1.0860000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.2989999999999999,"15min":1.077},"min":{"1min":0.69799999999999995,"5min":0.47899999999999998,"15min":0.40600000000000003},"max":{"1min":1.3160000000000001,"5min":28.881,"15min":28.881},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.2949999999999999,"15min":1.0629999999999999},"min":{"1min":0.69999999999999996,"5min":0.51500000000000001,"15min":0.38900000000000001},"max":{"1min":1.349,"5min":25.364000000000001,"15min":25.364000000000001},"last":0.84099999999999997}]},{"osd":3,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.2270000000000001,"15min":0.99399999999999999},"min":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.54000000000000004},"max":{"1min":1.907,"5min":26.553000000000001,"15min":26.553000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.3069999999999999,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.482,"5min":28.922000000000001,"15min":28.922000000000001},"last":1.1240000000000001}]},{"osd":4,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.333,"15min":1.143},"min":{"1min":0.70799999999999996,"5min":0.41699999999999998,"15min":0.255},"max":{"1min":1.5589999999999999,"5min":25.253,"15min":25.253},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.351,"15min":1.2010000000000001},"min":{"1min":0.85599999999999998,"5min":0.55500000000000005,"15min":0.28199999999999997},"max":{"1min":1.6739999999999999,"5min":27.076000000000001,"15min":27.076000000000001},"last":1.353}]},{"osd":5,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.4159999999999999,"15min":1.214},"min":{"1min":0.89300000000000002,"5min":0.442,"15min":0.34399999999999997},"max":{"1min":1.9650000000000001,"5min":26.440000000000001,"15min":26.440000000000001},"last":1.266},{"interface":"front","average":{"1min":1.296,"5min":1.2470000000000001,"15min":1.1930000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.32700000000000001},"max":{"1min":1.7070000000000001,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.4299999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:58:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4590000000000001,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.626,"15min":0.36499999999999999},"max":{"1min":1.9019999999999999,"5min":26.398,"15min":26.398},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.53,"15min":1.3340000000000001},"min":{"1min":1.123,"5min":0.58399999999999996,"15min":0.373},"max":{"1min":2.097,"5min":25.401,"15min":25.401},"last":1.3919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":291900,"kb_used_data":238516,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94079940,"statfs":{"total":96636764160,"available":96337858560,"internally_reserved":0,"allocated":244240384,"data_stored":239018754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78500000000000003},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1699999999999999,"5min":1.919,"15min":1.919},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.83099999999999996,"15min":0.85099999999999998},"min":{"1min":0.53600000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.3089999999999999,"5min":1.456,"15min":1.456},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.91000000000000003,"15min":0.84099999999999997},"min":{"1min":0.56899999999999995,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":1.105,"5min":11.834,"15min":11.834},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89300000000000002,"15min":0.89700000000000002},"min":{"1min":0.64900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2330000000000001,"5min":1.8029999999999999,"15min":2.0489999999999999},"last":1.2230000000000001}]},{"osd":2,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94199999999999995,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.46100000000000002},"max":{"1min":1.3300000000000001,"5min":2.028,"15min":2.028},"last":1.085},{"interface":"front","average":{"1min":1.081,"5min":1.038,"15min":0.996},"min":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.327,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.087,"15min":1.0349999999999999},"min":{"1min":0.77400000000000002,"5min":0.54400000000000004,"15min":0.53300000000000003},"max":{"1min":1.284,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.02,"15min":0.94899999999999995},"min":{"1min":0.72099999999999997,"5min":0.56200000000000006,"15min":0.503},"max":{"1min":1.1240000000000001,"5min":11.756,"15min":11.756},"last":0.89700000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.99099999999999999,"15min":1.014},"min":{"1min":0.63,"5min":0.31,"15min":0.31},"max":{"1min":1.254,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.22,"5min":1.224,"15min":1.1799999999999999},"min":{"1min":0.78800000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.724,"5min":4.056,"15min":4.056},"last":0.86399999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.2110000000000001,"15min":1.0860000000000001},"min":{"1min":0.72399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.581,"5min":20.446000000000002,"15min":20.446000000000002},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1559999999999999,"15min":1.1459999999999999},"min":{"1min":0.74299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4730000000000001,"5min":5.6369999999999996,"15min":5.6369999999999996},"last":1.03}]},{"osd":7,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2529999999999999,"15min":1.23},"min":{"1min":0.91400000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.665,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.861,"15min":1.4630000000000001},"min":{"1min":1.129,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.8360000000000001,"5min":50.406999999999996,"15min":50.406999999999996},"last":1.3260000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575015,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361084,"kb_used_data":293812,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94010756,"statfs":{"total":96636764160,"available":96267014144,"internally_reserved":0,"allocated":300863488,"data_stored":293092827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85799999999999998},"min":{"1min":0.55200000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0740000000000001,"5min":1.6970000000000001,"15min":3.3490000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.85899999999999999,"15min":0.91400000000000003},"min":{"1min":0.51800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1559999999999999,"5min":1.6379999999999999,"15min":3.4780000000000002},"last":0.86899999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.876,"15min":0.89100000000000001},"min":{"1min":0.58199999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1910000000000001,"5min":1.302,"15min":3.258},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94799999999999995,"15min":0.92900000000000005},"min":{"1min":0.63200000000000001,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.165,"5min":2.1259999999999999,"15min":3.4289999999999998},"last":0.78300000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91900000000000004,"15min":0.94199999999999995},"min":{"1min":0.54200000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3280000000000001,"5min":1.9139999999999999,"15min":3.3010000000000002},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.018,"5min":1.0529999999999999,"15min":1.0069999999999999},"min":{"1min":0.72499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4390000000000001,"5min":9.8599999999999994,"15min":9.8599999999999994},"last":0.91800000000000004}]},{"osd":3,"last update":"Mon Oct 30 12:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.876,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.504,"5min":1.5960000000000001,"15min":4.1070000000000002},"last":1.081},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.996,"15min":1.036},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.147,"5min":2.5190000000000001,"15min":3.524},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0069999999999999,"15min":1.05},"min":{"1min":0.63500000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.218,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.02},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.258,"15min":1.153},"min":{"1min":1.008,"5min":0.624,"15min":0.624},"max":{"1min":1.5680000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.3979999999999999}]},{"osd":6,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1699999999999999,"15min":1.137},"min":{"1min":0.83399999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.627,"5min":2.802,"15min":2.802},"last":1.2989999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1100000000000001,"15min":1.1120000000000001},"min":{"1min":0.70599999999999996,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2210000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.226}]},{"osd":7,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.1080000000000001,"15min":1.175},"min":{"1min":0.80000000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5069999999999999,"5min":2.3940000000000001,"15min":2.508},"last":1.252},{"interface":"front","average":{"1min":1.319,"5min":1.3149999999999999,"15min":1.256},"min":{"1min":1.0740000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.673,"5min":2.137,"15min":2.137},"last":1.4810000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413764,"kb_used_data":342332,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93958076,"statfs":{"total":96636764160,"available":96213069824,"internally_reserved":0,"allocated":350547968,"data_stored":342386739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4550000000000001,"5min":2.819,"15min":2.819},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.0149999999999999,"15min":0.92000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.39500000000000002},"max":{"1min":1.3759999999999999,"5min":2.859,"15min":2.859},"last":1.2809999999999999}]},{"osd":1,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.54700000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1659999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.92300000000000004,"15min":0.94399999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.288,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.0389999999999999}]},{"osd":2,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.48999999999999999},"max":{"1min":1.379,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.58099999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.353,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.026,"15min":0.98099999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.55700000000000005},"max":{"1min":1.4359999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.0069999999999999,"15min":1.0309999999999999},"min":{"1min":0.66600000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.27,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2010000000000001}]},{"osd":5,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0609999999999999,"15min":1.073},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.393,"5min":1.5369999999999999,"15min":2.0049999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0029999999999999,"15min":1.0249999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.262,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.083,"15min":1.1220000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2929999999999999,"5min":1.595,"15min":2.1360000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.1639999999999999,"15min":1.1859999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2669999999999999,"5min":1.661,"15min":2.093},"last":1.123}]},{"osd":7,"last update":"Mon Oct 30 12:58:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2509999999999999,"15min":1.2030000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.73199999999999998},"max":{"1min":1.4179999999999999,"5min":1.7130000000000001,"15min":1.7689999999999999},"last":1.25},{"interface":"front","average":{"1min":1.163,"5min":1.23,"15min":1.2629999999999999},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4370000000000001,"5min":1.716,"15min":1.851},"last":1.369}]}]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":402224,"kb_used_data":334952,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93969616,"statfs":{"total":96636764160,"available":96224886784,"internally_reserved":0,"allocated":342990848,"data_stored":334684047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68876744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.60299999999999998,"15min":0.504},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":11.262,"15min":11.262},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.63800000000000001,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.61099999999999999,"5min":11.244,"15min":11.244},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.53800000000000003,"15min":0.499},"min":{"1min":0.30299999999999999,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.66700000000000004,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.309},"max":{"1min":0.73899999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.35699999999999998},"max":{"1min":0.83199999999999996,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Oct 30 12:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.499},"max":{"1min":1.097,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2090000000000001,"5min":1.786,"15min":1.867},"last":0.67200000000000004}]},{"osd":5,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90700000000000003,"15min":0.86299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.03,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.92300000000000004},"min":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.60199999999999998},"max":{"1min":1.163,"5min":3.125,"15min":3.125},"last":0.84599999999999997}]},{"osd":6,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92800000000000005,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1699999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.745},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.871,"15min":0.876},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.167,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.79200000000000004}]},{"osd":7,"last update":"Mon Oct 30 12:58:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0669999999999999,"5min":1.8600000000000001,"15min":2.5830000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.88100000000000001,"15min":0.88600000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60099999999999998},"max":{"1min":1.1240000000000001,"5min":1.5700000000000001,"15min":2.1890000000000001},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":204268,"kb_used_data":150884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94167572,"statfs":{"total":96636764160,"available":96427593728,"internally_reserved":0,"allocated":154505216,"data_stored":149426315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.496},"min":{"1min":0.32100000000000001,"5min":0.251,"15min":0.22700000000000001},"max":{"1min":0.60499999999999998,"5min":0.73999999999999999,"15min":6.0140000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54200000000000004,"15min":0.66000000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.311},"max":{"1min":0.747,"5min":1.6020000000000001,"15min":34.262999999999998},"last":0.67900000000000005}]},{"osd":1,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.27400000000000002},"max":{"1min":0.63700000000000001,"5min":1.7529999999999999,"15min":3.0270000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":1.7969999999999999,"15min":6.0579999999999998},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70499999999999996,"5min":1.8500000000000001,"15min":7.4960000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.81200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.671,"15min":59.115000000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81799999999999995,"15min":0.875},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0609999999999999,"5min":1.9199999999999999,"15min":9.8170000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.95999999999999996},"min":{"1min":0.55200000000000005,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":1.0740000000000001,"5min":1.165,"15min":34.307000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84799999999999998,"15min":1.0429999999999999},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.262,"5min":1.8959999999999999,"15min":48.951000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.84799999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.175,"5min":1.827,"15min":27.899000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":1.004},"min":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.56799999999999995},"max":{"1min":1.1180000000000001,"5min":1.2689999999999999,"15min":34.350999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.86799999999999999,"15min":0.90100000000000002},"min":{"1min":0.623,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.169,"5min":1.9379999999999999,"15min":5.0949999999999998},"last":1.0760000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.88500000000000001,"15min":1.071},"min":{"1min":0.67500000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.111,"5min":1.337,"15min":34.420000000000002},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":1.165},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.59099999999999997},"max":{"1min":1.091,"5min":1.974,"15min":40.710000000000001},"last":1.2050000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274664,"kb_used_data":212960,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94097176,"statfs":{"total":96636764160,"available":96355508224,"internally_reserved":0,"allocated":218071040,"data_stored":212483944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.69199999999999995,"15min":0.64700000000000002},"min":{"1min":0.28100000000000003,"5min":0.24199999999999999,"15min":0.23100000000000001},"max":{"1min":1.008,"5min":26.349,"15min":26.349},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.251},"max":{"1min":1.264,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":0.65600000000000003,"5min":15.718999999999999,"15min":15.718999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.69299999999999995,"15min":0.71699999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92700000000000005,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.623}]},{"osd":3,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66700000000000004,"15min":0.73899999999999999},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":9.7469999999999999,"15min":9.7469999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57799999999999996,"15min":0.66200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32800000000000001},"max":{"1min":1.3220000000000001,"5min":2.4870000000000001,"15min":8.1620000000000008},"last":0.52800000000000002}]},{"osd":4,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.70399999999999996,"5min":0.59999999999999998,"15min":0.54600000000000004},"max":{"1min":1.4790000000000001,"5min":2.5190000000000001,"15min":3.0030000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":0.94999999999999996},"min":{"1min":0.621,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.044,"5min":15.653,"15min":15.653},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.90900000000000003,"15min":0.879},"min":{"1min":0.60599999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.999,"5min":9.8460000000000001,"15min":9.8460000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.97399999999999998,"15min":1.006},"min":{"1min":0.5,"5min":0.5,"15min":0.48099999999999998},"max":{"1min":1.2030000000000001,"5min":12.15,"15min":12.15},"last":0.94299999999999995}]},{"osd":6,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81100000000000005,"15min":0.92600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.427,"5min":2.4569999999999999,"15min":8.1080000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1,"15min":1.032},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2829999999999999,"5min":15.766999999999999,"15min":15.766999999999999},"last":1.0920000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0329999999999999},"min":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.57699999999999996},"max":{"1min":1.1279999999999999,"5min":12.098000000000001,"15min":12.098000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85499999999999998,"15min":0.97499999999999998},"min":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3740000000000001,"5min":2.5489999999999999,"15min":8.0510000000000002},"last":1.044}]}]},{"osd":0,"up_from":13,"seq":55834575014,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":377664,"kb_used_data":310392,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93994176,"statfs":{"total":96636764160,"available":96250036224,"internally_reserved":0,"allocated":317841408,"data_stored":309508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.24199999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.5880000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27800000000000002},"max":{"1min":0.73199999999999998,"5min":1.732,"15min":1.732},"last":0.65700000000000003}]},{"osd":2,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.738},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":1.556,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.79000000000000004,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.36099999999999999},"max":{"1min":1.516,"5min":1.851,"15min":1.851},"last":0.71699999999999997}]},{"osd":4,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.91200000000000003,"15min":0.86499999999999999},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4419999999999999,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":1.1279999999999999,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.93300000000000005,"15min":0.89800000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.441},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.704},"last":1},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.85999999999999999},"min":{"1min":0.58299999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.29,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.095}]},{"osd":6,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.88300000000000001,"15min":0.90000000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.163,"5min":1.639,"15min":1.903},"last":1.038},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.84499999999999997,"15min":0.86699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.254,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0660000000000001}]},{"osd":7,"last update":"Mon Oct 30 12:58:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90700000000000003,"15min":0.89200000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.4079999999999999,"5min":2.9409999999999998,"15min":2.9409999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.94399999999999995,"15min":0.91500000000000004},"min":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.52200000000000002},"max":{"1min":1.1890000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.147}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37765120,"data_stored":37749396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41967616,"data_stored":41943984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12972032,"data_stored":12946424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":42512384,"data_stored":42488633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61665292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35987456,"data_stored":35965743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":51695616,"data_stored":51675052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":71319552,"data_stored":71303896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278876160,"data_stored":270866944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175493120,"data_stored":170237247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140922880,"data_stored":136177178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":299868160,"data_stored":291892701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":288256000,"data_stored":280418734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":264265728,"data_stored":256824371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":191934464,"data_stored":187040989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":262901760,"data_stored":255370516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T12:58:48.574 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-30T12:58:48.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 Processor -- start 2023-10-30T12:58:48.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 -- start start 2023-10-30T12:58:48.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0448151a20 0x7f04480cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04481581f0 con 0x7f04480c74a0 2023-10-30T12:58:48.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04480cb9f0 con 0x7f04480c1020 2023-10-30T12:58:48.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f044d2c9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04481576d0 con 0x7f0448151a20 2023-10-30T12:58:48.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f04477fe640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f04480c74a0 0x7f0448161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:57762/0 (socket says 172.21.15.8:57762) 2023-10-30T12:58:48.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f04477fe640 1 -- 172.21.15.8:0/3252976289 learned_addr learned my addr 172.21.15.8:0/3252976289 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:48.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f0446ffd640 1 --2- 172.21.15.8:0/3252976289 >> v2:172.21.15.46:3300/0 conn(0x7f0448151a20 0x7f04480cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675503215 0 0) 0x7f04481581f0 con 0x7f04480c74a0 2023-10-30T12:58:48.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0434003530 con 0x7f04480c74a0 2023-10-30T12:58:48.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f0446ffd640 1 -- 172.21.15.8:0/3252976289 >> v1:172.21.15.46:6789/0 conn(0x7f04480c1020 legacy=0x7f0448072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f0446ffd640 1 -- 172.21.15.8:0/3252976289 >> v1:172.21.15.8:6789/0 conn(0x7f04480c74a0 legacy=0x7f0448161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873674480 0 0) 0x7f0434003530 con 0x7f04480c74a0 2023-10-30T12:58:48.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.710+0000 7f0446ffd640 1 -- 172.21.15.8:0/3252976289 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448151e00 con 0x7f0448151a20 2023-10-30T12:58:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f0446ffd640 1 --2- 172.21.15.8:0/3252976289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f04480cb4b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04380097c0 tx=0x7f043801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998bacfddbe86477 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 <== mon.1 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440001a00 con 0x7f0448151a20 2023-10-30T12:58:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 <== mon.1 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04380a8610 con 0x7f0448151a20 2023-10-30T12:58:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f04467fc640 1 -- 172.21.15.8:0/3252976289 <== mon.1 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04380a8980 con 0x7f0448151a20 2023-10-30T12:58:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 -- 172.21.15.8:0/3252976289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 msgr2=0x7f04480cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 --2- 172.21.15.8:0/3252976289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f04480cb4b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04380097c0 tx=0x7f043801eb90 comp rx=0 tx=0).stop 2023-10-30T12:58:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 -- 172.21.15.8:0/3252976289 shutdown_connections 2023-10-30T12:58:48.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 --2- 172.21.15.8:0/3252976289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f04480cb4b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 -- 172.21.15.8:0/3252976289 >> 172.21.15.8:0/3252976289 conn(0x7f044806da40 msgr2=0x7f044806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:48.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 -- 172.21.15.8:0/3252976289 shutdown_connections 2023-10-30T12:58:48.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.714+0000 7f044d2c9640 1 -- 172.21.15.8:0/3252976289 wait complete. 2023-10-30T12:58:48.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 Processor -- start 2023-10-30T12:58:48.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- start start 2023-10-30T12:58:48.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f04480c74a0 0x7f04480c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f0448149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480c9790 con 0x7f04480c1020 2023-10-30T12:58:48.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480c8ea0 con 0x7f0448151a20 2023-10-30T12:58:48.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04481ef5b0 con 0x7f04480c74a0 2023-10-30T12:58:48.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41794/0 (socket says 172.21.15.8:41794) 2023-10-30T12:58:48.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 -- 172.21.15.8:0/1451339373 learned_addr learned my addr 172.21.15.8:0/1451339373 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:48.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f043ffff640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f04480c74a0 0x7f04480c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f04477fe640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f0448149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 -- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f04480c74a0 msgr2=0x7f04480c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f04480c74a0 0x7f04480c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 -- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 msgr2=0x7f0448149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f0448149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 -- 172.21.15.8:0/1451339373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009420 con 0x7f04480c1020 2023-10-30T12:58:48.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f0446ffd640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f043801b7a0 tx=0x7f043801b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04380b0850 con 0x7f04480c1020 2023-10-30T12:58:48.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04380b09b0 con 0x7f04480c1020 2023-10-30T12:58:48.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04480c9140 con 0x7f04480c1020 2023-10-30T12:58:48.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04380b0d40 con 0x7f04480c1020 2023-10-30T12:58:48.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.718+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04481e1b60 con 0x7f04480c1020 2023-10-30T12:58:48.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044804e170 con 0x7f04480c1020 2023-10-30T12:58:48.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f04380b7040 con 0x7f04480c1020 2023-10-30T12:58:48.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f043f7fe640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 0x7f0428068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:48.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f0438121400 con 0x7f04480c1020 2023-10-30T12:58:48.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f043ffff640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 0x7f0428068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:48.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.722+0000 7f043ffff640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 0x7f0428068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f04480cbdf0 tx=0x7f0430009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:48.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.726+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f04380ee5d0 con 0x7f04480c1020 2023-10-30T12:58:48.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.882+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f04480cafe0 con 0x7f04480c1020 2023-10-30T12:58:48.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.886+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0438006e20 con 0x7f04480c1020 2023-10-30T12:58:48.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f043f7fe640 1 -- 172.21.15.8:0/1451339373 <== mon.0 v2:172.21.15.8:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f04380ee5d0 con 0x7f04480c1020 2023-10-30T12:58:48.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 msgr2=0x7f0428068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 0x7f0428068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f04480cbdf0 tx=0x7f0430009040 comp rx=0 tx=0).stop 2023-10-30T12:58:48.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 msgr2=0x7f0448072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:48.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f043801b7a0 tx=0x7f043801b930 comp rx=0 tx=0).stop 2023-10-30T12:58:48.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 shutdown_connections 2023-10-30T12:58:48.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f04280664a0 0x7f0428068960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0448151a20 0x7f0448149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f04480c74a0 0x7f04480c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 --2- 172.21.15.8:0/1451339373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f04480c1020 0x7f0448072300 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:48.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 >> 172.21.15.8:0/1451339373 conn(0x7f044806da40 msgr2=0x7f04480c0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:48.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 shutdown_connections 2023-10-30T12:58:48.899 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:48.890+0000 7f044d2c9640 1 -- 172.21.15.8:0/1451339373 wait complete. 2023-10-30T12:58:48.915 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-30T12:58:48.916 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:58:49.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 Processor -- start 2023-10-30T12:58:49.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 -- start start 2023-10-30T12:58:49.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85e415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85e40c42a0 0x7f85e40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e4159f20 con 0x7f85e40c4a10 2023-10-30T12:58:49.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e415dec0 con 0x7f85e40c42a0 2023-10-30T12:58:49.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85eb53e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41599a0 con 0x7f85e4153c80 2023-10-30T12:58:49.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85e8ad9640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f85e40c4a10 0x7f85e415a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:57764/0 (socket says 172.21.15.8:57764) 2023-10-30T12:58:49.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85e8ad9640 1 -- 172.21.15.8:0/3281022753 learned_addr learned my addr 172.21.15.8:0/3281022753 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:49.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85e9adb640 1 --2- 172.21.15.8:0/3281022753 >> v2:172.21.15.46:3300/0 conn(0x7f85e40c42a0 0x7f85e40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85e92da640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85e415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559223131 0 0) 0x7f85e4159f20 con 0x7f85e40c4a10 2023-10-30T12:58:49.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.054+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d0003530 con 0x7f85e40c4a10 2023-10-30T12:58:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3076380471 0 0) 0x7f85d0003530 con 0x7f85e40c4a10 2023-10-30T12:58:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 msgr2=0x7f85e415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85e415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.067 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 >> v2:172.21.15.46:3300/0 conn(0x7f85e40c42a0 msgr2=0x7f85e40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 --2- 172.21.15.8:0/3281022753 >> v2:172.21.15.46:3300/0 conn(0x7f85e40c42a0 0x7f85e40ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e415d620 con 0x7f85e40c4a10 2023-10-30T12:58:49.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f85d4001680 con 0x7f85e40c4a10 2023-10-30T12:58:49.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e9adb640 1 --2- 172.21.15.8:0/3281022753 >> v2:172.21.15.46:3300/0 conn(0x7f85e40c42a0 0x7f85e40ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-30T12:58:49.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 >> v1:172.21.15.8:6789/0 conn(0x7f85e40c4a10 legacy=0x7f85e415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85d0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85d0003f00 0x7f85d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d0004d10 0x7f85d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415dec0 con 0x7f85d0003f00 2023-10-30T12:58:49.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41599a0 con 0x7f85d0004d10 2023-10-30T12:58:49.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d000b9c0 con 0x7f85e4153c80 2023-10-30T12:58:49.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85d0003f00 0x7f85d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e92da640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85d0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e9adb640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d0004d10 0x7f85d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 -- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 msgr2=0x7f85d0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85e4153c80 0x7f85d0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 -- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d0004d10 msgr2=0x7f85d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d0004d10 0x7f85d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 -- 172.21.15.8:0/3281022753 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0003530 con 0x7f85d0003f00 2023-10-30T12:58:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e9adb640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d0004d10 0x7f85d0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.058+0000 7f85e8ad9640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85d0003f00 0x7f85d00047d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f85e40c4e30 tx=0x7f85d401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0c43aa622d33cd server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40a5910 con 0x7f85d0003f00 2023-10-30T12:58:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f85d40a5a70 con 0x7f85d0003f00 2023-10-30T12:58:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85dbfff640 1 -- 172.21.15.8:0/3281022753 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40a5e40 con 0x7f85d0003f00 2023-10-30T12:58:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- 172.21.15.8:0/3281022753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85d0003f00 msgr2=0x7f85d00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 --2- 172.21.15.8:0/3281022753 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85d0003f00 0x7f85d00047d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f85e40c4e30 tx=0x7f85d401cca0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85e9adb640 1 -- 172.21.15.8:0/3281022753 reap_dead start 2023-10-30T12:58:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- 172.21.15.8:0/3281022753 shutdown_connections 2023-10-30T12:58:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- 172.21.15.8:0/3281022753 >> 172.21.15.8:0/3281022753 conn(0x7f85e40715c0 msgr2=0x7f85e40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- 172.21.15.8:0/3281022753 shutdown_connections 2023-10-30T12:58:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- 172.21.15.8:0/3281022753 wait complete. 2023-10-30T12:58:49.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 Processor -- start 2023-10-30T12:58:49.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- start start 2023-10-30T12:58:49.080 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85d0003f00 0x7f85e41f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e415d0e0 0x7f85e4156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e414ac30 con 0x7f85e4155840 2023-10-30T12:58:49.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40c50b0 con 0x7f85e415d0e0 2023-10-30T12:58:49.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85eb53e640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4154a00 con 0x7f85d0003f00 2023-10-30T12:58:49.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85e9adb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85e9adb640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41806/0 (socket says 172.21.15.8:41806) 2023-10-30T12:58:49.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.062+0000 7f85e9adb640 1 -- 172.21.15.8:0/2358240666 learned_addr learned my addr 172.21.15.8:0/2358240666 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:49.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e92da640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85d0003f00 0x7f85e41f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e8ad9640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e415d0e0 0x7f85e4156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 -- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85d0003f00 msgr2=0x7f85e41f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85d0003f00 0x7f85e41f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 -- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e415d0e0 msgr2=0x7f85e4156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e415d0e0 0x7f85e4156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 -- 172.21.15.8:0/2358240666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4154440 con 0x7f85e4155840 2023-10-30T12:58:49.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85e9adb640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85e4159ae0 tx=0x7f85dc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00e070 con 0x7f85e4155840 2023-10-30T12:58:49.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f85dc011e10 con 0x7f85e4155840 2023-10-30T12:58:49.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0039a0 con 0x7f85e4155840 2023-10-30T12:58:49.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e4154720 con 0x7f85e4155840 2023-10-30T12:58:49.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e4156990 con 0x7f85e4155840 2023-10-30T12:58:49.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e404e170 con 0x7f85e4155840 2023-10-30T12:58:49.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.066+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85dc003b00 con 0x7f85e4155840 2023-10-30T12:58:49.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.070+0000 7f85d9ffb640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 0x7f85c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.070+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f85dc084a10 con 0x7f85e4155840 2023-10-30T12:58:49.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.070+0000 7f85e92da640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 0x7f85c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.070+0000 7f85e92da640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 0x7f85c8068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009b00 tx=0x7f85e0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.074+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f85dc051a30 con 0x7f85e4155840 2023-10-30T12:58:49.266 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e40b52c0 con 0x7f85e4155840 2023-10-30T12:58:49.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85d9ffb640 1 -- 172.21.15.8:0/2358240666 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f85dc053960 con 0x7f85e4155840 2023-10-30T12:58:49.268 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:49.268 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5688,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/1050115493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":1050115493},{"type":"v1","addr":"172.21.15.8:6838","nonce":1050115493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:29.814420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:active","state_seq":31,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:58:49.269 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 65 2023-10-30T12:58:49.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 msgr2=0x7f85c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 0x7f85c8068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009b00 tx=0x7f85e0009190 comp rx=0 tx=0).stop 2023-10-30T12:58:49.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 msgr2=0x7f85e40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85e4159ae0 tx=0x7f85dc0115a0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 shutdown_connections 2023-10-30T12:58:49.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f85c80664a0 0x7f85c8068960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e415d0e0 0x7f85e4156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85e4155840 0x7f85e40ce420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 --2- 172.21.15.8:0/2358240666 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f85d0003f00 0x7f85e41f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 >> 172.21.15.8:0/2358240666 conn(0x7f85e40715c0 msgr2=0x7f85e41589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 shutdown_connections 2023-10-30T12:58:49.273 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.262+0000 7f85eb53e640 1 -- 172.21.15.8:0/2358240666 wait complete. 2023-10-30T12:58:49.289 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-30T12:58:49.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 Processor -- start 2023-10-30T12:58:49.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 -- start start 2023-10-30T12:58:49.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f527006b760 0x7f527015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f527006bb40 0x7f5270157a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5270071e00 con 0x7f527006c420 2023-10-30T12:58:49.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5270070020 con 0x7f527006b760 2023-10-30T12:58:49.434 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f5275b94640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527015eb90 con 0x7f527006bb40 2023-10-30T12:58:49.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526f7fe640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f527006b760 0x7f527015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526ffff640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f527006c420 0x7f52700c57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60952/0 (socket says 172.21.15.8:60952) 2023-10-30T12:58:49.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526ffff640 1 -- 172.21.15.8:0/1658125598 learned_addr learned my addr 172.21.15.8:0/1658125598 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526effd640 1 --2- 172.21.15.8:0/1658125598 >> v2:172.21.15.46:3300/0 conn(0x7f527006bb40 0x7f5270157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526f7fe640 1 -- 172.21.15.8:0/1658125598 >> v2:172.21.15.46:3300/0 conn(0x7f527006bb40 msgr2=0x7f5270157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526f7fe640 1 --2- 172.21.15.8:0/1658125598 >> v2:172.21.15.46:3300/0 conn(0x7f527006bb40 0x7f5270157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526f7fe640 1 -- 172.21.15.8:0/1658125598 >> v1:172.21.15.8:6789/0 conn(0x7f527006c420 legacy=0x7f52700c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526f7fe640 1 -- 172.21.15.8:0/1658125598 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270070f10 con 0x7f527006b760 2023-10-30T12:58:49.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.426+0000 7f526e7fc640 1 -- 172.21.15.8:0/1658125598 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453391209 0 0) 0x7f5270071e00 con 0x7f527006c420 2023-10-30T12:58:49.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f526f7fe640 1 --2- 172.21.15.8:0/1658125598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006b760 0x7f527015e650 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f527006fa40 tx=0x7f526001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d73d427e9408d server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f526e7fc640 1 -- 172.21.15.8:0/1658125598 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600a7070 con 0x7f527006b760 2023-10-30T12:58:49.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f526e7fc640 1 -- 172.21.15.8:0/1658125598 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f52600ab430 con 0x7f527006b760 2023-10-30T12:58:49.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f526e7fc640 1 -- 172.21.15.8:0/1658125598 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600ab7e0 con 0x7f527006b760 2023-10-30T12:58:49.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 -- 172.21.15.8:0/1658125598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006b760 msgr2=0x7f527015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 --2- 172.21.15.8:0/1658125598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006b760 0x7f527015e650 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f527006fa40 tx=0x7f526001c430 comp rx=0 tx=0).stop 2023-10-30T12:58:49.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 -- 172.21.15.8:0/1658125598 shutdown_connections 2023-10-30T12:58:49.440 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 --2- 172.21.15.8:0/1658125598 >> v2:172.21.15.46:3300/0 conn(0x7f527006bb40 0x7f5270157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 --2- 172.21.15.8:0/1658125598 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006b760 0x7f527015e650 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 -- 172.21.15.8:0/1658125598 >> 172.21.15.8:0/1658125598 conn(0x7f52700cac70 msgr2=0x7f52700cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 -- 172.21.15.8:0/1658125598 shutdown_connections 2023-10-30T12:58:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.430+0000 7f5275b94640 1 -- 172.21.15.8:0/1658125598 wait complete. 2023-10-30T12:58:49.442 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 Processor -- start 2023-10-30T12:58:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- start start 2023-10-30T12:58:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f527006b760 0x7f527014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527006c420 0x7f52700ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700c9bf0 con 0x7f527006bb40 2023-10-30T12:58:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700c4130 con 0x7f527006c420 2023-10-30T12:58:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52700c4920 con 0x7f527006b760 2023-10-30T12:58:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55224/0 (socket says 172.21.15.8:55224) 2023-10-30T12:58:49.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 -- 172.21.15.8:0/344819133 learned_addr learned my addr 172.21.15.8:0/344819133 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526ffff640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527006c420 0x7f52700ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526f7fe640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f527006b760 0x7f527014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 -- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f527006b760 msgr2=0x7f527014b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f527006b760 0x7f527014b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 -- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527006c420 msgr2=0x7f52700ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527006c420 0x7f52700ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 -- 172.21.15.8:0/344819133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009420 con 0x7f527006bb40 2023-10-30T12:58:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f526effd640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f526400e900 tx=0x7f526400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264009690 con 0x7f527006bb40 2023-10-30T12:58:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f526400ca00 con 0x7f527006bb40 2023-10-30T12:58:49.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400cdb0 con 0x7f527006bb40 2023-10-30T12:58:49.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52700c4cf0 con 0x7f527006bb40 2023-10-30T12:58:49.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.434+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52700c36b0 con 0x7f527006bb40 2023-10-30T12:58:49.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527004e170 con 0x7f527006bb40 2023-10-30T12:58:49.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5264013470 con 0x7f527006bb40 2023-10-30T12:58:49.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f5257fff640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 0x7f525c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f5264084260 con 0x7f527006bb40 2023-10-30T12:58:49.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f526f7fe640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 0x7f525c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.438+0000 7f526f7fe640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 0x7f525c068960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f52600af000 tx=0x7f52600a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.442+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f52640513a0 con 0x7f527006bb40 2023-10-30T12:58:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.630+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52700c4cf0 con 0x7f527006bb40 2023-10-30T12:58:49.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.630+0000 7f5257fff640 1 -- 172.21.15.8:0/344819133 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f52700c4cf0 con 0x7f527006bb40 2023-10-30T12:58:49.638 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:49.638 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4217977183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4217977183},{"type":"v1","addr":"172.21.15.46:6836","nonce":4217977183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3564454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3564454356},{"type":"v1","addr":"172.21.15.46:6837","nonce":3564454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5688,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6838/1050115493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6835","nonce":1050115493},{"type":"v1","addr":"172.21.15.8:6838","nonce":1050115493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-30T12:45:18.463017+0000","modified":"2023-10-30T12:57:29.814420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5266,"mds_1":5535,"mds_2":5394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"f","rank":0,"incarnation":41,"state":"up:active","state_seq":31,"addr":"172.21.15.46:6833/1405231055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1405231055},{"type":"v1","addr":"172.21.15.46:6833","nonce":1405231055}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5394":{"gid":5394,"name":"a","rank":2,"incarnation":52,"state":"up:active","state_seq":35,"addr":"172.21.15.8:6837/1849048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":1849048959},{"type":"v1","addr":"172.21.15.8:6837","nonce":1849048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5535":{"gid":5535,"name":"c","rank":1,"incarnation":60,"state":"up:active","state_seq":31,"addr":"172.21.15.8:6839/4033248661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":4033248661},{"type":"v1","addr":"172.21.15.8:6839","nonce":4033248661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-30T12:58:49.639 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 65 2023-10-30T12:58:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 msgr2=0x7f525c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 0x7f525c068960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f52600af000 tx=0x7f52600a7000 comp rx=0 tx=0).stop 2023-10-30T12:58:49.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 msgr2=0x7f52700ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f526400e900 tx=0x7f526400edc0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 shutdown_connections 2023-10-30T12:58:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f525c0664a0 0x7f525c068960 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527006c420 0x7f52700ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f527006bb40 0x7f52700ca010 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 --2- 172.21.15.8:0/344819133 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f527006b760 0x7f527014b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 >> 172.21.15.8:0/344819133 conn(0x7f52700cac70 msgr2=0x7f52700c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 shutdown_connections 2023-10-30T12:58:49.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.634+0000 7f5275b94640 1 -- 172.21.15.8:0/344819133 wait complete. 2023-10-30T12:58:49.659 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T12:58:49.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 Processor -- start 2023-10-30T12:58:49.803 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 -- start start 2023-10-30T12:58:49.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7fe8151c20 0x7f7fe80c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7fe80cb8b0 0x7f7fe806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80c5030 con 0x7f7fe8151840 2023-10-30T12:58:49.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe8157830 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fecf94640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe8157b10 con 0x7f7fe8151c20 2023-10-30T12:58:49.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7fe80cb8b0 0x7f7fe806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7fe80cb8b0 0x7f7fe806b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55232/0 (socket says 172.21.15.8:55232) 2023-10-30T12:58:49.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/1663054520 learned_addr learned my addr 172.21.15.8:0/1663054520 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:49.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe659c640 1 --1- 172.21.15.8:0/1663054520 >> v1:172.21.15.8:6789/0 conn(0x7f7fe8151840 0x7f7fe8072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60960/0 (socket says 172.21.15.8:60960) 2023-10-30T12:58:49.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe5d9b640 1 --2- 172.21.15.8:0/1663054520 >> v2:172.21.15.46:3300/0 conn(0x7f7fe8151c20 0x7f7fe80c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/1663054520 >> v2:172.21.15.46:3300/0 conn(0x7f7fe8151c20 msgr2=0x7f7fe80c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:49.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 --2- 172.21.15.8:0/1663054520 >> v2:172.21.15.46:3300/0 conn(0x7f7fe8151c20 0x7f7fe80c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/1663054520 >> v1:172.21.15.8:6789/0 conn(0x7f7fe8151840 legacy=0x7f7fe8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/1663054520 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8157df0 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.798+0000 7f7fe559a640 1 -- 172.21.15.8:0/1663054520 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590783624 0 0) 0x7f7fe80c5030 con 0x7f7fe8151840 2023-10-30T12:58:49.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fe6d9d640 1 --2- 172.21.15.8:0/1663054520 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe80cb8b0 0x7f7fe806b600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7fe806d4d0 tx=0x7f7fe001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16276e4a9500eb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fe559a640 1 -- 172.21.15.8:0/1663054520 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00af400 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fe559a640 1 -- 172.21.15.8:0/1663054520 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7fe00ab050 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fe559a640 1 -- 172.21.15.8:0/1663054520 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00af7a0 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 -- 172.21.15.8:0/1663054520 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe80cb8b0 msgr2=0x7f7fe806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 --2- 172.21.15.8:0/1663054520 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe80cb8b0 0x7f7fe806b600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7fe806d4d0 tx=0x7f7fe001f1c0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 -- 172.21.15.8:0/1663054520 shutdown_connections 2023-10-30T12:58:49.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 --2- 172.21.15.8:0/1663054520 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe80cb8b0 0x7f7fe806b600 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 --2- 172.21.15.8:0/1663054520 >> v2:172.21.15.46:3300/0 conn(0x7f7fe8151c20 0x7f7fe80c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 -- 172.21.15.8:0/1663054520 >> 172.21.15.8:0/1663054520 conn(0x7f7fe80cac70 msgr2=0x7f7fe80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 -- 172.21.15.8:0/1663054520 shutdown_connections 2023-10-30T12:58:49.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.802+0000 7f7fecf94640 1 -- 172.21.15.8:0/1663054520 wait complete. 2023-10-30T12:58:49.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 Processor -- start 2023-10-30T12:58:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- start start 2023-10-30T12:58:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7fe80cb8b0 0x7f7fe8155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe8151840 0x7f7fe81dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81f2cb0 con 0x7f7fe8151c20 2023-10-30T12:58:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81e1330 con 0x7f7fe8151840 2023-10-30T12:58:49.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81ee7c0 con 0x7f7fe80cb8b0 2023-10-30T12:58:49.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55236/0 (socket says 172.21.15.8:55236) 2023-10-30T12:58:49.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/264718336 learned_addr learned my addr 172.21.15.8:0/264718336 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe659c640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7fe80cb8b0 0x7f7fe8155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe5d9b640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe8151840 0x7f7fe81dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7fe80cb8b0 msgr2=0x7f7fe8155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7fe80cb8b0 0x7f7fe8155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe8151840 msgr2=0x7f7fe81dfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe8151840 0x7f7fe81dfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 -- 172.21.15.8:0/264718336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0009b30 con 0x7f7fe8151c20 2023-10-30T12:58:49.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fe6d9d640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7fe001cdf0 tx=0x7f7fe001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00afdc0 con 0x7f7fe8151c20 2023-10-30T12:58:49.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7fe0009510 con 0x7f7fe8151c20 2023-10-30T12:58:49.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00ba6a0 con 0x7f7fe8151c20 2023-10-30T12:58:49.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe81eea60 con 0x7f7fe8151c20 2023-10-30T12:58:49.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.806+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe81f2410 con 0x7f7fe8151c20 2023-10-30T12:58:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8066510 con 0x7f7fe8151c20 2023-10-30T12:58:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7fe00b6040 con 0x7f7fe8151c20 2023-10-30T12:58:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fd6ffd640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 0x7f7fc8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f7fe0122530 con 0x7f7fe8151c20 2023-10-30T12:58:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fe659c640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 0x7f7fc8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.810+0000 7f7fe659c640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 0x7f7fc8068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000f70 tx=0x7f7fd8007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.814+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7fe00ef5e0 con 0x7f7fe8151c20 2023-10-30T12:58:49.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.978+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fe81eebf0 con 0x7f7fe8151c20 2023-10-30T12:58:49.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.978+0000 7f7fd6ffd640 1 -- 172.21.15.8:0/264718336 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17471 (secure 0 0 0) 0x7f7fe00f2c00 con 0x7f7fe8151c20 2023-10-30T12:58:49.985 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-30T12:58:49.985 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":53,"fsid":"d8d9e729-1294-467e-ac5d-e55aafd29b99","created":"2023-10-30T12:44:52.332905+0000","modified":"2023-10-30T12:57:07.233484+0000","last_up_change":"2023-10-30T12:45:04.370812+0000","last_in_change":"2023-10-30T12:44:55.596742+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T12:45:05.671325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-30T12:45:16.140516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-30T12:45:17.462529+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a2dea52-4b4c-49b4-87ed-1b82e4a35088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6801","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6805","nonce":2064691129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6809","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6811","nonce":2064691129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6823","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6827","nonce":2064691129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6815","nonce":2064691129},{"type":"v1","addr":"172.21.15.8:6819","nonce":2064691129}]},"public_addr":"172.21.15.8:6805/2064691129","cluster_addr":"172.21.15.8:6811/2064691129","heartbeat_back_addr":"172.21.15.8:6827/2064691129","heartbeat_front_addr":"172.21.15.8:6819/2064691129","state":["exists","up"]},{"osd":1,"uuid":"86288470-58d9-44da-a82b-62eb5d96c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6807","nonce":2525970788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6816","nonce":2525970788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6831","nonce":2525970788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2525970788},{"type":"v1","addr":"172.21.15.8:6824","nonce":2525970788}]},"public_addr":"172.21.15.8:6807/2525970788","cluster_addr":"172.21.15.8:6816/2525970788","heartbeat_back_addr":"172.21.15.8:6831/2525970788","heartbeat_front_addr":"172.21.15.8:6824/2525970788","state":["exists","up"]},{"osd":2,"uuid":"89d39136-f1b7-4184-8d24-66bb377177ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6803","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6808","nonce":713612811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6813","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6817","nonce":713612811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6829","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6830","nonce":713612811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":713612811},{"type":"v1","addr":"172.21.15.8:6826","nonce":713612811}]},"public_addr":"172.21.15.8:6808/713612811","cluster_addr":"172.21.15.8:6817/713612811","heartbeat_back_addr":"172.21.15.8:6830/713612811","heartbeat_front_addr":"172.21.15.8:6826/713612811","state":["exists","up"]},{"osd":3,"uuid":"1d51b4c3-4a06-4f22-9a48-440692bba9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6802","nonce":469140225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6810","nonce":469140225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6821","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6825","nonce":469140225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":469140225},{"type":"v1","addr":"172.21.15.8:6818","nonce":469140225}]},"public_addr":"172.21.15.8:6802/469140225","cluster_addr":"172.21.15.8:6810/469140225","heartbeat_back_addr":"172.21.15.8:6825/469140225","heartbeat_front_addr":"172.21.15.8:6818/469140225","state":["exists","up"]},{"osd":4,"uuid":"abceded1-2ec0-4352-bb15-4eb891564cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6812","nonce":4276465018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6815","nonce":4276465018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6821","nonce":4276465018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":4276465018},{"type":"v1","addr":"172.21.15.46:6818","nonce":4276465018}]},"public_addr":"172.21.15.46:6812/4276465018","cluster_addr":"172.21.15.46:6815/4276465018","heartbeat_back_addr":"172.21.15.46:6821/4276465018","heartbeat_front_addr":"172.21.15.46:6818/4276465018","state":["exists","up"]},{"osd":5,"uuid":"7b7814ee-3a48-48e1-8add-ad5f8090df2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6801","nonce":3519076799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6803","nonce":3519076799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6807","nonce":3519076799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3519076799},{"type":"v1","addr":"172.21.15.46:6805","nonce":3519076799}]},"public_addr":"172.21.15.46:6801/3519076799","cluster_addr":"172.21.15.46:6803/3519076799","heartbeat_back_addr":"172.21.15.46:6807/3519076799","heartbeat_front_addr":"172.21.15.46:6805/3519076799","state":["exists","up"]},{"osd":6,"uuid":"3898a740-c477-4f12-b71b-a60cdf7bd407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6825","nonce":815955451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6827","nonce":815955451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6831","nonce":815955451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":815955451},{"type":"v1","addr":"172.21.15.46:6829","nonce":815955451}]},"public_addr":"172.21.15.46:6825/815955451","cluster_addr":"172.21.15.46:6827/815955451","heartbeat_back_addr":"172.21.15.46:6831/815955451","heartbeat_front_addr":"172.21.15.46:6829/815955451","state":["exists","up"]},{"osd":7,"uuid":"d0befb09-63a8-4ab8-8464-efdf1c8fc231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6809","nonce":1029552049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6814","nonce":1029552049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6823","nonce":1029552049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":1029552049},{"type":"v1","addr":"172.21.15.46:6820","nonce":1029552049}]},"public_addr":"172.21.15.46:6809/1029552049","cluster_addr":"172.21.15.46:6814/1029552049","heartbeat_back_addr":"172.21.15.46:6823/1029552049","heartbeat_front_addr":"172.21.15.46:6820/1029552049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.222667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.195330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.375209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.259197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.070828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.158998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T12:45:01.148591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6838/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.8:6839/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.8:6837/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/3916953811":"2023-10-31T12:55:12.201822+0000","172.21.15.46:6833/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/3033347051":"2023-10-31T12:57:07.233391+0000","172.21.15.46:6832/3888673612":"2023-10-31T12:50:57.121966+0000","172.21.15.8:6835/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6838/2992596118":"2023-10-31T12:49:27.084924+0000","172.21.15.8:6839/710307090":"2023-10-31T12:47:22.043669+0000","172.21.15.8:6834/2926864154":"2023-10-31T12:53:02.155598+0000","172.21.15.8:6836/710307090":"2023-10-31T12:47:22.043669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T12:58:49.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 msgr2=0x7f7fc8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 0x7f7fc8068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000f70 tx=0x7f7fd8007b60 comp rx=0 tx=0).stop 2023-10-30T12:58:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 msgr2=0x7f7fe81eeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:49.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7fe001cdf0 tx=0x7f7fe001bf60 comp rx=0 tx=0).stop 2023-10-30T12:58:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 shutdown_connections 2023-10-30T12:58:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f7fc80664a0 0x7f7fc8068960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe8151840 0x7f7fe81dfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7fe8151c20 0x7f7fe81eeef0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 --2- 172.21.15.8:0/264718336 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f7fe80cb8b0 0x7f7fe8155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:49.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 >> 172.21.15.8:0/264718336 conn(0x7f7fe80cac70 msgr2=0x7f7fe80c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:49.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 shutdown_connections 2023-10-30T12:58:49.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:49.982+0000 7f7fecf94640 1 -- 172.21.15.8:0/264718336 wait complete. 2023-10-30T12:58:50.006 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-30T12:58:50.150 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 Processor -- start 2023-10-30T12:58:50.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 -- start start 2023-10-30T12:58:50.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00800c4a10 0x7f008015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0080159f20 con 0x7f0080153c80 2023-10-30T12:58:50.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008015d620 con 0x7f00800c42a0 2023-10-30T12:58:50.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f00849c4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008015dec0 con 0x7f00800c4a10 2023-10-30T12:58:50.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007e59c640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f0080153c80 0x7f008015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60976/0 (socket says 172.21.15.8:60976) 2023-10-30T12:58:50.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007e59c640 1 -- 172.21.15.8:0/1928824110 learned_addr learned my addr 172.21.15.8:0/1928824110 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:50.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007dd9b640 1 --2- 172.21.15.8:0/1928824110 >> v2:172.21.15.46:3300/0 conn(0x7f00800c4a10 0x7f008015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531890353 0 0) 0x7f0080159f20 con 0x7f0080153c80 2023-10-30T12:58:50.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0068003530 con 0x7f0080153c80 2023-10-30T12:58:50.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355381971 0 0) 0x7f0068003530 con 0x7f0080153c80 2023-10-30T12:58:50.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 >> v2:172.21.15.46:3300/0 conn(0x7f00800c4a10 msgr2=0x7f008015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:50.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 --2- 172.21.15.8:0/1928824110 >> v2:172.21.15.46:3300/0 conn(0x7f00800c4a10 0x7f008015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 >> v1:172.21.15.46:6789/0 conn(0x7f00800c42a0 legacy=0x7f00800ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008015d900 con 0x7f0080153c80 2023-10-30T12:58:50.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1327636587 0 0) 0x7f0070001680 con 0x7f0080153c80 2023-10-30T12:58:50.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 >> v1:172.21.15.8:6789/0 conn(0x7f0080153c80 legacy=0x7f008015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f0068003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f00680047d0 0x7f0068004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068005190 0x7f006800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008015dec0 con 0x7f0068003980 2023-10-30T12:58:50.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006800c040 con 0x7f0068005190 2023-10-30T12:58:50.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006800c200 con 0x7f00680047d0 2023-10-30T12:58:50.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007e59c640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f0068003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007ed9d640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068005190 0x7f006800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.146+0000 7f007dd9b640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f00680047d0 0x7f0068004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 -- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f00680047d0 msgr2=0x7f0068004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:50.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f00680047d0 0x7f0068004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 -- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068005190 msgr2=0x7f006800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:50.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068005190 0x7f006800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 -- 172.21.15.8:0/1928824110 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068003530 con 0x7f0068003980 2023-10-30T12:58:50.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007ed9d640 1 -- 172.21.15.8:0/1928824110 reap_dead start 2023-10-30T12:58:50.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007e59c640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f0068003da0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f007000c240 tx=0x7f007001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad951353f5fe5186 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070005ae0 con 0x7f0068003980 2023-10-30T12:58:50.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f00700ab070 con 0x7f0068003980 2023-10-30T12:58:50.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f007d59a640 1 -- 172.21.15.8:0/1928824110 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070007070 con 0x7f0068003980 2023-10-30T12:58:50.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 -- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 msgr2=0x7f0068003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f0068003da0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f007000c240 tx=0x7f007001f3b0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 -- 172.21.15.8:0/1928824110 shutdown_connections 2023-10-30T12:58:50.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 --2- 172.21.15.8:0/1928824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f0068003da0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 -- 172.21.15.8:0/1928824110 >> 172.21.15.8:0/1928824110 conn(0x7f00800715c0 msgr2=0x7f00800719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:50.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 -- 172.21.15.8:0/1928824110 shutdown_connections 2023-10-30T12:58:50.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.150+0000 7f00849c4640 1 -- 172.21.15.8:0/1928824110 wait complete. 2023-10-30T12:58:50.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 Processor -- start 2023-10-30T12:58:50.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- start start 2023-10-30T12:58:50.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0080157810 0x7f0080144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00801e9200 0x7f0080149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00801f3600 con 0x7f0068003980 2023-10-30T12:58:50.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0080144700 con 0x7f00801e9200 2023-10-30T12:58:50.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00801449e0 con 0x7f0080157810 2023-10-30T12:58:50.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55256/0 (socket says 172.21.15.8:55256) 2023-10-30T12:58:50.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 -- 172.21.15.8:0/3875909403 learned_addr learned my addr 172.21.15.8:0/3875909403 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:50.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007dd9b640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0080157810 0x7f0080144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007ed9d640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00801e9200 0x7f0080149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 -- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0080157810 msgr2=0x7f0080144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0080157810 0x7f0080144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 -- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00801e9200 msgr2=0x7f0080149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00801e9200 0x7f0080149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 -- 172.21.15.8:0/3875909403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080146a90 con 0x7f0068003980 2023-10-30T12:58:50.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f007e59c640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f007001f1e0 tx=0x7f0068005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070001640 con 0x7f0068003980 2023-10-30T12:58:50.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0070007070 con 0x7f0068003980 2023-10-30T12:58:50.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00800c5bc0 con 0x7f0068003980 2023-10-30T12:58:50.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070007440 con 0x7f0068003980 2023-10-30T12:58:50.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.154+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0080147030 con 0x7f0068003980 2023-10-30T12:58:50.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008004e170 con 0x7f0068003980 2023-10-30T12:58:50.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f00700b9030 con 0x7f0068003980 2023-10-30T12:58:50.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f006effd640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 0x7f0060068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6370+0+0 (secure 0 0 0) 0x7f00700b7070 con 0x7f0068003980 2023-10-30T12:58:50.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f007dd9b640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 0x7f0060068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.158+0000 7f007dd9b640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 0x7f0060068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0080148ea0 tx=0x7f0074008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.162+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f00700ee410 con 0x7f0068003980 2023-10-30T12:58:50.358 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.350+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f00800cad40 con 0x7f0068003980 2023-10-30T12:58:50.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.354+0000 7f006effd640 1 -- 172.21.15.8:0/3875909403 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f00700f1a30 con 0x7f0068003980 2023-10-30T12:58:50.361 INFO:teuthology.orchestra.run.smithi008.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-30T12:58:50.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 msgr2=0x7f0060068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 0x7f0060068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0080148ea0 tx=0x7f0074008040 comp rx=0 tx=0).stop 2023-10-30T12:58:50.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 msgr2=0x7f00801f2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f007001f1e0 tx=0x7f0068005800 comp rx=0 tx=0).stop 2023-10-30T12:58:50.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 shutdown_connections 2023-10-30T12:58:50.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f00600664a0 0x7f0060068960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00801e9200 0x7f0080149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f0080157810 0x7f0080144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 --2- 172.21.15.8:0/3875909403 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0068003980 0x7f00801f2600 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 >> 172.21.15.8:0/3875909403 conn(0x7f00800715c0 msgr2=0x7f00800c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:50.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 shutdown_connections 2023-10-30T12:58:50.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.358+0000 7f00849c4640 1 -- 172.21.15.8:0/3875909403 wait complete. 2023-10-30T12:58:50.387 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-30T12:58:50.433 INFO:tasks.ceph.mds.a.smithi008.stdout:starting mds.a at 2023-10-30T12:58:50.435 INFO:tasks.ceph.mds.c.smithi008.stdout:starting mds.c at 2023-10-30T12:58:50.461 INFO:tasks.ceph.mds.f.smithi046.stdout:starting mds.f at 2023-10-30T12:58:50.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 Processor -- start 2023-10-30T12:58:50.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 -- start start 2023-10-30T12:58:50.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 0x7f56f415a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f4153f80 con 0x7f56f40c4810 2023-10-30T12:58:50.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f4159df0 con 0x7f56f40c4410 2023-10-30T12:58:50.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56fa5c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f415a7c0 con 0x7f56f4153ba0 2023-10-30T12:58:50.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 0x7f56f415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3301/0 says I am v2:172.21.15.8:39768/0 (socket says 172.21.15.8:39768) 2023-10-30T12:58:50.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f8b62640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f56f40c4810 0x7f56f415cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.8:55732/0 (socket says 172.21.15.8:55732) 2023-10-30T12:58:50.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 -- 172.21.15.8:0/2645198771 learned_addr learned my addr 172.21.15.8:0/2645198771 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:50.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 0x7f56f415a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.8:52148/0 (socket says 172.21.15.8:52148) 2023-10-30T12:58:50.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 -- 172.21.15.8:0/2645198771 >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 msgr2=0x7f56f415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-30T12:58:50.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 --2- 172.21.15.8:0/2645198771 >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 0x7f56f415a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 -- 172.21.15.8:0/2645198771 >> v1:172.21.15.46:6789/0 conn(0x7f56f40c4810 legacy=0x7f56f415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.534+0000 7f56f3fff640 1 -- 172.21.15.8:0/2645198771 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f41597c0 con 0x7f56f4153ba0 2023-10-30T12:58:50.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56f3fff640 1 --2- 172.21.15.8:0/2645198771 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009870 tx=0x7f56e401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad95be2dcdc95f server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56f2ffd640 1 -- 172.21.15.8:0/2645198771 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40ab680 con 0x7f56f4153ba0 2023-10-30T12:58:50.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56f2ffd640 1 -- 172.21.15.8:0/2645198771 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56e40ab7e0 con 0x7f56f4153ba0 2023-10-30T12:58:50.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56f2ffd640 1 -- 172.21.15.8:0/2645198771 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40abb90 con 0x7f56f4153ba0 2023-10-30T12:58:50.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2645198771 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 msgr2=0x7f56f40cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2645198771 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009870 tx=0x7f56e401c480 comp rx=0 tx=0).stop 2023-10-30T12:58:50.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2645198771 shutdown_connections 2023-10-30T12:58:50.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2645198771 >> v2:172.21.15.46:3300/0 conn(0x7f56f40c4410 0x7f56f415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2645198771 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f4153ba0 0x7f56f40cb4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2645198771 >> 172.21.15.8:0/2645198771 conn(0x7f56f4071790 msgr2=0x7f56f4071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:50.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2645198771 shutdown_connections 2023-10-30T12:58:50.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.538+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2645198771 wait complete. 2023-10-30T12:58:50.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 Processor -- start 2023-10-30T12:58:50.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- start start 2023-10-30T12:58:50.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f40c4410 0x7f56f415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56f40c4810 0x7f56f40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f415c080 con 0x7f56f4153ba0 2023-10-30T12:58:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40c6490 con 0x7f56f40c4810 2023-10-30T12:58:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40c6cb0 con 0x7f56f40c4410 2023-10-30T12:58:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55286/0 (socket says 172.21.15.8:55286) 2023-10-30T12:58:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 -- 172.21.15.8:0/2785889912 learned_addr learned my addr 172.21.15.8:0/2785889912 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f3fff640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f40c4410 0x7f56f415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f37fe640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56f40c4810 0x7f56f40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 -- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f40c4410 msgr2=0x7f56f415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f40c4410 0x7f56f415c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 -- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56f40c4810 msgr2=0x7f56f40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:50.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56f40c4810 0x7f56f40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:50.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 -- 172.21.15.8:0/2785889912 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40094d0 con 0x7f56f4153ba0 2023-10-30T12:58:50.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f8b62640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c7a0 tx=0x7f56ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00a640 con 0x7f56f4153ba0 2023-10-30T12:58:50.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56ec00a7a0 con 0x7f56f4153ba0 2023-10-30T12:58:50.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f40c66f0 con 0x7f56f4153ba0 2023-10-30T12:58:50.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00ab30 con 0x7f56f4153ba0 2023-10-30T12:58:50.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.542+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f415bd10 con 0x7f56f4153ba0 2023-10-30T12:58:50.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f415c230 con 0x7f56f4153ba0 2023-10-30T12:58:50.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56ec00ac90 con 0x7f56f4153ba0 2023-10-30T12:58:50.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56f0ff9640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 0x7f56e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:50.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6628+0+0 (secure 0 0 0) 0x7f56ec0839b0 con 0x7f56f4153ba0 2023-10-30T12:58:50.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56f3fff640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 0x7f56e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:50.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.546+0000 7f56f3fff640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 0x7f56e0068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f56e40af000 tx=0x7f56e40b6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:50.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.550+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56f415c230 con 0x7f56f4153ba0 2023-10-30T12:58:50.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:50.746+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56f4151820 con 0x7f56f4153ba0 2023-10-30T12:58:51.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56f0ff9640 1 -- 172.21.15.8:0/2785889912 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56ec050b30 con 0x7f56f4153ba0 2023-10-30T12:58:51.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 msgr2=0x7f56e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 0x7f56e0068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f56e40af000 tx=0x7f56e40b6040 comp rx=0 tx=0).stop 2023-10-30T12:58:51.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 msgr2=0x7f56f41552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c7a0 tx=0x7f56ec00cc60 comp rx=0 tx=0).stop 2023-10-30T12:58:51.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 shutdown_connections 2023-10-30T12:58:51.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f56e00664a0 0x7f56e0068960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56f40c4810 0x7f56f40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56f4153ba0 0x7f56f41552c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 --2- 172.21.15.8:0/2785889912 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f56f40c4410 0x7f56f415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 >> 172.21.15.8:0/2785889912 conn(0x7f56f4071790 msgr2=0x7f56f4151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:51.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 shutdown_connections 2023-10-30T12:58:51.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.362+0000 7f56fa5c5640 1 -- 172.21.15.8:0/2785889912 wait complete. 2023-10-30T12:58:51.388 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-30T12:58:51.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 Processor -- start 2023-10-30T12:58:51.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 -- start start 2023-10-30T12:58:51.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8454151730 0x7f84540c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:51.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8454157620 con 0x7f8454151b10 2023-10-30T12:58:51.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8454072bd0 con 0x7f8454072740 2023-10-30T12:58:51.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845c29c640 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8454158560 con 0x7f8454151730 2023-10-30T12:58:51.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8454151730 0x7f84540c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:51.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8454151730 0x7f84540c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55288/0 (socket says 172.21.15.8:55288) 2023-10-30T12:58:51.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 -- 172.21.15.8:0/2811310496 learned_addr learned my addr 172.21.15.8:0/2811310496 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:51.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f8459837640 1 --1- 172.21.15.8:0/2811310496 >> v1:172.21.15.8:6789/0 conn(0x7f8454151b10 0x7f84540c5fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60988/0 (socket says 172.21.15.8:60988) 2023-10-30T12:58:51.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 -- 172.21.15.8:0/2811310496 >> v1:172.21.15.46:6789/0 conn(0x7f8454072740 legacy=0x7f845406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 -- 172.21.15.8:0/2811310496 >> v1:172.21.15.8:6789/0 conn(0x7f8454151b10 legacy=0x7f84540c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 -- 172.21.15.8:0/2811310496 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84541578e0 con 0x7f8454151730 2023-10-30T12:58:51.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f8459036640 1 -- 172.21.15.8:0/2811310496 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074086823 0 0) 0x7f8454157620 con 0x7f8454151b10 2023-10-30T12:58:51.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f845a038640 1 --2- 172.21.15.8:0/2811310496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84540c57e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8450009870 tx=0x7f845001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df39f38d2e89f0a server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:51.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.534+0000 7f8459036640 1 -- 172.21.15.8:0/2811310496 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500a8a30 con 0x7f8454151730 2023-10-30T12:58:51.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f8459036640 1 -- 172.21.15.8:0/2811310496 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f84500a8b90 con 0x7f8454151730 2023-10-30T12:58:51.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f8459036640 1 -- 172.21.15.8:0/2811310496 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500b0440 con 0x7f8454151730 2023-10-30T12:58:51.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- 172.21.15.8:0/2811310496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 msgr2=0x7f84540c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 --2- 172.21.15.8:0/2811310496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84540c57e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8450009870 tx=0x7f845001ed30 comp rx=0 tx=0).stop 2023-10-30T12:58:51.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- 172.21.15.8:0/2811310496 shutdown_connections 2023-10-30T12:58:51.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 --2- 172.21.15.8:0/2811310496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84540c57e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- 172.21.15.8:0/2811310496 >> 172.21.15.8:0/2811310496 conn(0x7f845406e150 msgr2=0x7f8454069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:51.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- 172.21.15.8:0/2811310496 shutdown_connections 2023-10-30T12:58:51.546 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- 172.21.15.8:0/2811310496 wait complete. 2023-10-30T12:58:51.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 Processor -- start 2023-10-30T12:58:51.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- start start 2023-10-30T12:58:51.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 0x7f84540c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:51.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:51.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8454151b10 0x7f8454147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:51.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540c34c0 con 0x7f8454151730 2023-10-30T12:58:51.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540c2600 con 0x7f8454072740 2023-10-30T12:58:51.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845c29c640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540c3710 con 0x7f8454151b10 2023-10-30T12:58:51.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f8459837640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:51.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f8459837640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55300/0 (socket says 172.21.15.8:55300) 2023-10-30T12:58:51.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f8459837640 1 -- 172.21.15.8:0/3359320380 learned_addr learned my addr 172.21.15.8:0/3359320380 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:51.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845a038640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 0x7f84540c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:51.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.538+0000 7f845a839640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8454151b10 0x7f8454147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:51.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 -- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8454151b10 msgr2=0x7f8454147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8454151b10 0x7f8454147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 -- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 msgr2=0x7f84540c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:51.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 0x7f84540c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:51.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 -- 172.21.15.8:0/3359320380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84500094d0 con 0x7f8454151730 2023-10-30T12:58:51.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f845a038640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 0x7f84540c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:51.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f8459837640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f844400c4a0 tx=0x7f844400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:51.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440074e0 con 0x7f8454151730 2023-10-30T12:58:51.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8444007640 con 0x7f8454151730 2023-10-30T12:58:51.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84540c39f0 con 0x7f8454151730 2023-10-30T12:58:51.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444004a50 con 0x7f8454151730 2023-10-30T12:58:51.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8454152720 con 0x7f8454151730 2023-10-30T12:58:51.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845404e170 con 0x7f8454151730 2023-10-30T12:58:51.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8444004bb0 con 0x7f8454151730 2023-10-30T12:58:51.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 0x7f8438068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:51.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6628+0+0 (secure 0 0 0) 0x7f8444083810 con 0x7f8454151730 2023-10-30T12:58:51.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.542+0000 7f845a038640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 0x7f8438068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:51.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.546+0000 7f845a038640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 0x7f8438068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f84500b2000 tx=0x7f8450008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:51.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.546+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8444050850 con 0x7f8454151730 2023-10-30T12:58:51.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:51.710+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f845414f2b0 con 0x7f8454151730 2023-10-30T12:58:52.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.366+0000 7f844affd640 1 -- 172.21.15.8:0/3359320380 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8444004e60 con 0x7f8454151730 2023-10-30T12:58:52.373 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' removed 2023-10-30T12:58:52.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 msgr2=0x7f8438068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 0x7f8438068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f84500b2000 tx=0x7f8450008040 comp rx=0 tx=0).stop 2023-10-30T12:58:52.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 msgr2=0x7f84541476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f844400c4a0 tx=0x7f844400c960 comp rx=0 tx=0).stop 2023-10-30T12:58:52.376 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 shutdown_connections 2023-10-30T12:58:52.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f84380664a0 0x7f8438068960 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f84500b2000 tx=0x7f8450008040 comp rx=0 tx=0).stop 2023-10-30T12:58:52.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f8454151b10 0x7f8454147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8454151730 0x7f84541476c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 --2- 172.21.15.8:0/3359320380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8454072740 0x7f84540c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 >> 172.21.15.8:0/3359320380 conn(0x7f845406e150 msgr2=0x7f84540caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:52.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 shutdown_connections 2023-10-30T12:58:52.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.370+0000 7f845c29c640 1 -- 172.21.15.8:0/3359320380 wait complete. 2023-10-30T12:58:52.407 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-30T12:58:52.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 Processor -- start 2023-10-30T12:58:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 -- start start 2023-10-30T12:58:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f5480071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5480070b10 con 0x7f54800c3630 2023-10-30T12:58:52.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5480071970 con 0x7f548006dfc0 2023-10-30T12:58:52.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f548652a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800c7d30 con 0x7f548006dbe0 2023-10-30T12:58:52.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f5484ac7640 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f54800c3630 0x7f548014f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:32768/0 (socket says 172.21.15.8:32768) 2023-10-30T12:58:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f5484ac7640 1 -- 172.21.15.8:0/1374021751 learned_addr learned my addr 172.21.15.8:0/1374021751 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-30T12:58:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547ffff640 1 --2- 172.21.15.8:0/1374021751 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f5480071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121970775 0 0) 0x7f5480070b10 con 0x7f54800c3630 2023-10-30T12:58:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5468003450 con 0x7f54800c3630 2023-10-30T12:58:52.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 <== mon.1 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794995986 0 0) 0x7f5480071970 con 0x7f548006dfc0 2023-10-30T12:58:52.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547ffff640 1 -- 172.21.15.8:0/1374021751 >> v1:172.21.15.46:6789/0 conn(0x7f548006dfc0 legacy=0x7f54801567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547ffff640 1 -- 172.21.15.8:0/1374021751 >> v1:172.21.15.8:6789/0 conn(0x7f54800c3630 legacy=0x7f548014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.558+0000 7f547ffff640 1 -- 172.21.15.8:0/1374021751 --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800c3ac0 con 0x7f548006dbe0 2023-10-30T12:58:52.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f547ffff640 1 --2- 172.21.15.8:0/1374021751 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f5480071e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54700097f0 tx=0x7f547001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb965374a636d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:52.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 <== mon.2 v2:172.21.15.46:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700a89b0 con 0x7f548006dbe0 2023-10-30T12:58:52.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 <== mon.2 v2:172.21.15.46:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f54700a8b10 con 0x7f548006dbe0 2023-10-30T12:58:52.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f547effd640 1 -- 172.21.15.8:0/1374021751 <== mon.2 v2:172.21.15.46:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700a8e80 con 0x7f548006dbe0 2023-10-30T12:58:52.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 -- 172.21.15.8:0/1374021751 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 msgr2=0x7f5480071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 --2- 172.21.15.8:0/1374021751 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f5480071e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54700097f0 tx=0x7f547001ecb0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 -- 172.21.15.8:0/1374021751 shutdown_connections 2023-10-30T12:58:52.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 --2- 172.21.15.8:0/1374021751 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f5480071e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 -- 172.21.15.8:0/1374021751 >> 172.21.15.8:0/1374021751 conn(0x7f54800c7fa0 msgr2=0x7f54800c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:52.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 -- 172.21.15.8:0/1374021751 shutdown_connections 2023-10-30T12:58:52.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.562+0000 7f548652a640 1 -- 172.21.15.8:0/1374021751 wait complete. 2023-10-30T12:58:52.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 Processor -- start 2023-10-30T12:58:52.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- start start 2023-10-30T12:58:52.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 --2- >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f54800c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:52.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 0x7f548014b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:52.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:52.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548014cc40 con 0x7f54800c3630 2023-10-30T12:58:52.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54801ef850 con 0x7f548006dfc0 2023-10-30T12:58:52.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- --> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54801efaa0 con 0x7f548006dbe0 2023-10-30T12:58:52.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:52.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55304/0 (socket says 172.21.15.8:55304) 2023-10-30T12:58:52.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 -- 172.21.15.8:0/1241463452 learned_addr learned my addr 172.21.15.8:0/1241463452 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-30T12:58:52.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547ffff640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f54800c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:52.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547f7fe640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 0x7f548014b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:52.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 -- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 msgr2=0x7f54800c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f54800c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 -- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 msgr2=0x7f548014b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 0x7f548014b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 -- 172.21.15.8:0/1241463452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009420 con 0x7f54800c3630 2023-10-30T12:58:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547f7fe640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 0x7f548014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-30T12:58:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f5484ac7640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f547800caa0 tx=0x7f547800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547800ac00 con 0x7f54800c3630 2023-10-30T12:58:52.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f547800ad60 con 0x7f54800c3630 2023-10-30T12:58:52.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54800ca470 con 0x7f54800c3630 2023-10-30T12:58:52.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5478012640 con 0x7f54800c3630 2023-10-30T12:58:52.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54800caa30 con 0x7f54800c3630 2023-10-30T12:58:52.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480067610 con 0x7f54800c3630 2023-10-30T12:58:52.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5478007070 con 0x7f54800c3630 2023-10-30T12:58:52.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 0x7f546c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-30T12:58:52.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f5478083f00 con 0x7f54800c3630 2023-10-30T12:58:52.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547ffff640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 0x7f546c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-30T12:58:52.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.566+0000 7f547ffff640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 0x7f546c068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54700b2000 tx=0x7f54700aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-30T12:58:52.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.570+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5478051240 con 0x7f54800c3630 2023-10-30T12:58:52.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:52.730+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54800ca2d0 con 0x7f54800c3630 2023-10-30T12:58:53.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f547cff9640 1 -- 172.21.15.8:0/1241463452 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f54800ca2d0 con 0x7f54800c3630 2023-10-30T12:58:53.375 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' removed 2023-10-30T12:58:53.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 msgr2=0x7f546c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:53.377 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 0x7f546c068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54700b2000 tx=0x7f54700aa070 comp rx=0 tx=0).stop 2023-10-30T12:58:53.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 msgr2=0x7f548014e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-30T12:58:53.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f547800caa0 tx=0x7f547800cf60 comp rx=0 tx=0).stop 2023-10-30T12:58:53.378 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 shutdown_connections 2023-10-30T12:58:53.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:6832/2382673595,v1:172.21.15.8:6833/2382673595] conn(0x7f546c0664d0 0x7f546c068990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:53.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f548006dfc0 0x7f548014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:53.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f54800c3630 0x7f548014e020 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:53.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 --2- 172.21.15.8:0/1241463452 >> [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] conn(0x7f548006dbe0 0x7f54800c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-30T12:58:53.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 >> 172.21.15.8:0/1241463452 conn(0x7f54800c7fa0 msgr2=0x7f54801e0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-30T12:58:53.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 shutdown_connections 2023-10-30T12:58:53.381 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-30T12:58:53.374+0000 7f548652a640 1 -- 172.21.15.8:0/1241463452 wait complete. 2023-10-30T12:58:53.416 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-30T12:58:53.416 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-30T12:58:53.417 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.434 INFO:tasks.ceph.mds.a:Stopped 2023-10-30T12:58:53.434 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-30T12:58:53.435 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.533 INFO:tasks.ceph.mds.c:Stopped 2023-10-30T12:58:53.533 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-30T12:58:53.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.562 INFO:tasks.ceph.mds.e:Stopped 2023-10-30T12:58:53.563 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-30T12:58:53.563 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.576 INFO:tasks.ceph.mds.b:Stopped 2023-10-30T12:58:53.576 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-30T12:58:53.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.632 INFO:tasks.ceph.mds.d:Stopped 2023-10-30T12:58:53.633 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-30T12:58:53.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.693 INFO:tasks.ceph.mds.f:Stopped 2023-10-30T12:58:53.694 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-30T12:58:53.694 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-30T12:58:53.694 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:53.783 INFO:tasks.ceph.osd.0:Stopped 2023-10-30T12:58:53.784 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-30T12:58:53.784 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:58.711 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-30T12:58:59.886 INFO:tasks.ceph.osd.1:Stopped 2023-10-30T12:58:59.887 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-30T12:58:59.887 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:58:59.973 INFO:tasks.ceph.osd.2:Stopped 2023-10-30T12:58:59.974 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-30T12:58:59.974 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:04.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-30T12:59:04.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-10-30T12:59:06.077 INFO:tasks.ceph.osd.3:Stopped 2023-10-30T12:59:06.078 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-30T12:59:06.078 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:06.168 INFO:tasks.ceph.osd.4:Stopped 2023-10-30T12:59:06.169 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-30T12:59:06.169 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:06.251 INFO:tasks.ceph.osd.5:Stopped 2023-10-30T12:59:06.252 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-30T12:59:06.253 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:06.353 INFO:tasks.ceph.osd.6:Stopped 2023-10-30T12:59:06.354 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-30T12:59:06.354 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:10.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-30T12:59:10.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-30T12:59:12.456 INFO:tasks.ceph.osd.7:Stopped 2023-10-30T12:59:12.456 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-30T12:59:12.458 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-30T12:59:12.458 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:12.518 INFO:tasks.ceph.mgr.y:Stopped 2023-10-30T12:59:12.519 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-30T12:59:12.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:12.548 INFO:tasks.ceph.mgr.x:Stopped 2023-10-30T12:59:12.549 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-30T12:59:12.549 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T12:59:12.550 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:12.570 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T12:59:12.570 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T12:59:12.571 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:12.606 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T12:59:12.607 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T12:59:12.607 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T12:59:12.673 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T12:59:12.674 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-30T12:59:12.674 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | head -n 1 2023-10-30T12:59:12.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:59:12.717 DEBUG:teuthology.orchestra.run.smithi008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-30T12:59:13.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:59:13.089 DEBUG:teuthology.orchestra.run.smithi008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-30T12:59:13.393 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:59:13.393 DEBUG:teuthology.orchestra.run.smithi008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-30T12:59:13.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:59:13.549 DEBUG:teuthology.orchestra.run.smithi008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-30T12:59:13.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:59:13.712 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-30T12:59:13.920 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:59:13.920 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-30T12:59:14.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:59:14.134 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-30T12:59:14.298 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:59:14.298 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-30T12:59:14.566 INFO:tasks.ceph:Archiving mon data... 2023-10-30T12:59:14.567 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/data/mon.a.tgz 2023-10-30T12:59:14.568 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2023-10-30T12:59:14.578 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.T4b1mzUt26 2023-10-30T12:59:14.579 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.T4b1mzUt26 2023-10-30T12:59:14.711 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.T4b1mzUt26 2023-10-30T12:59:14.787 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.T4b1mzUt26 is 330KB 2023-10-30T12:59:14.855 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.T4b1mzUt26 2023-10-30T12:59:14.864 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/data/mon.b.tgz 2023-10-30T12:59:14.865 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-10-30T12:59:14.871 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.mGGsv5aTGz 2023-10-30T12:59:14.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mGGsv5aTGz 2023-10-30T12:59:14.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.mGGsv5aTGz 2023-10-30T12:59:15.060 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.mGGsv5aTGz is 334KB 2023-10-30T12:59:15.120 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.mGGsv5aTGz 2023-10-30T12:59:15.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-30T12:59:15.126 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/data/mon.c.tgz 2023-10-30T12:59:15.127 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-10-30T12:59:15.175 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.1Yxtpd6Wmh 2023-10-30T12:59:15.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1Yxtpd6Wmh 2023-10-30T12:59:15.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.1Yxtpd6Wmh 2023-10-30T12:59:15.368 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.1Yxtpd6Wmh is 334KB 2023-10-30T12:59:15.384 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.1Yxtpd6Wmh 2023-10-30T12:59:15.389 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-30T12:59:15.390 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T12:59:15.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T12:59:15.445 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-30T12:59:15.446 DEBUG:teuthology.orchestra.run.smithi008:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T12:59:15.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T12:59:15.491 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-30T12:59:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-30T12:59:15.511 INFO:tasks.ceph:Archiving crash dumps... 2023-10-30T12:59:15.512 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi008/crash 2023-10-30T12:59:15.514 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T12:59:15.563 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi046/crash 2023-10-30T12:59:15.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T12:59:15.575 INFO:tasks.ceph:Shutting down logrotate 2023-10-30T12:59:15.576 DEBUG:teuthology.orchestra.run.smithi008:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:59:15.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-30T12:59:15.667 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-30T12:59:15.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-30T12:59:15.714 INFO:tasks.ceph:Compressing logs... 2023-10-30T12:59:15.714 DEBUG:teuthology.orchestra.run.smithi008:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T12:59:15.736 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T12:59:15.754 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2023-10-30T12:59:15.755 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2023-10-30T12:59:15.756 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2023-10-30T12:59:15.757 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2023-10-30T12:59:15.757 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43126.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2023-10-30T12:59:15.758 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2023-10-30T12:59:15.758 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49498.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2023-10-30T12:59:15.759 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2023-10-30T12:59:15.760 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42944.log 2023-10-30T12:59:15.760 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2023-10-30T12:59:15.762 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42918.log 2023-10-30T12:59:15.763 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2023-10-30T12:59:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42944.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42944.log.gz 2023-10-30T12:59:15.763 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2023-10-30T12:59:15.764 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49997.log 2023-10-30T12:59:15.764 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42918.log.gz 2023-10-30T12:59:15.765 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47435.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz -5 2023-10-30T12:59:15.765 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49699.log 2023-10-30T12:59:15.766 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2023-10-30T12:59:15.766 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49997.log.gz 2023-10-30T12:59:15.767 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49699.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2023-10-30T12:59:15.767 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2023-10-30T12:59:15.768 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2023-10-30T12:59:15.768 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49371.log 2023-10-30T12:59:15.769 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51102.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2023-10-30T12:59:15.769 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38035.log 2023-10-30T12:59:15.770 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51310.log 2023-10-30T12:59:15.770 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49371.log.gz 2023-10-30T12:59:15.771 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.38035.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-30T12:59:15.771 INFO:teuthology.orchestra.run.smithi008.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.38035.log.gz 2023-10-30T12:59:15.772 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51743.log 2023-10-30T12:59:15.772 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2023-10-30T12:59:15.773 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44420.log 2023-10-30T12:59:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51743.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51743.log.gz 2023-10-30T12:59:15.774 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47183.log 2023-10-30T12:59:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.0.44420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44420.log.gz 2023-10-30T12:59:15.775 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2023-10-30T12:59:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47183.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47183.log.gz 2023-10-30T12:59:15.776 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2023-10-30T12:59:15.777 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49054.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2023-10-30T12:59:15.778 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2023-10-30T12:59:15.781 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2023-10-30T12:59:15.782 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2023-10-30T12:59:15.782 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2023-10-30T12:59:15.782 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44124.log 2023-10-30T12:59:15.783 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47622.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz 2023-10-30T12:59:15.783 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2023-10-30T12:59:15.783 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44124.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44124.log.gz 2023-10-30T12:59:15.784 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2023-10-30T12:59:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2023-10-30T12:59:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-30T12:59:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38939.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-30T12:59:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38867.log 2023-10-30T12:59:15.789 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2023-10-30T12:59:15.789 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2023-10-30T12:59:15.789 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2023-10-30T12:59:15.790 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38867.log.gz 2023-10-30T12:59:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38915.log 2023-10-30T12:59:15.790 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2023-10-30T12:59:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-30T12:59:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38843.log: /var/log/ceph/ceph-client.admin.38915.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-30T12:59:15.791 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38915.log.gz 2023-10-30T12:59:15.791 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2023-10-30T12:59:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2023-10-30T12:59:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T12:59:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-30T12:59:15.793 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38819.log 2023-10-30T12:59:15.793 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T12:59:15.793 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38819.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38819.log.gz 2023-10-30T12:59:15.793 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-30T12:59:15.801 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T12:59:15.801 INFO:teuthology.orchestra.run.smithi046.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-30T12:59:15.801 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-10-30T12:59:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-30T12:59:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.b.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-30T12:59:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-30T12:59:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.b.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-30T12:59:15.803 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-30T12:59:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38771.log 2023-10-30T12:59:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38891.log 2023-10-30T12:59:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38771.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38771.log.gz 2023-10-30T12:59:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38891.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38891.log.gz 2023-10-30T12:59:15.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50972.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2023-10-30T12:59:15.808 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2023-10-30T12:59:15.808 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-30T12:59:15.808 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2023-10-30T12:59:15.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50264.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2023-10-30T12:59:15.809 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45478.log 2023-10-30T12:59:15.809 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2023-10-30T12:59:15.810 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45880.log 2023-10-30T12:59:15.810 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45478.log.gz 2023-10-30T12:59:15.810 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49591.log 2023-10-30T12:59:15.811 INFO:teuthology.orchestra.run.smithi008.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2023-10-30T12:59:15.811 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45880.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45880.log.gz 2023-10-30T12:59:15.811 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47245.log 2023-10-30T12:59:15.812 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49591.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.49591.log.gz 2023-10-30T12:59:15.812 INFO:teuthology.orchestra.run.smithi008.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2023-10-30T12:59:15.812 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47245.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47245.log.gz 2023-10-30T12:59:15.812 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43531.log 2023-10-30T12:59:15.813 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49081.log 2023-10-30T12:59:15.813 INFO:teuthology.orchestra.run.smithi008.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2023-10-30T12:59:15.813 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43531.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43531.log.gz 2023-10-30T12:59:15.813 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43121.log 2023-10-30T12:59:15.814 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43915.log 2023-10-30T12:59:15.814 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49081.log.gz 2023-10-30T12:59:15.814 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43121.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43121.log.gz 2023-10-30T12:59:15.815 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2023-10-30T12:59:15.815 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50023.log 2023-10-30T12:59:15.815 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43915.log.gz 2023-10-30T12:59:15.815 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44867.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2023-10-30T12:59:15.816 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2023-10-30T12:59:15.816 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50023.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.50023.log.gz -5 2023-10-30T12:59:15.816 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49565.log 2023-10-30T12:59:15.817 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51206.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2023-10-30T12:59:15.817 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2023-10-30T12:59:15.817 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2023-10-30T12:59:15.817 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49565.log.gz 2023-10-30T12:59:15.818 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49784.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2023-10-30T12:59:15.818 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2023-10-30T12:59:15.818 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2023-10-30T12:59:15.819 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2023-10-30T12:59:15.819 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50698.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2023-10-30T12:59:15.819 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44555.log 2023-10-30T12:59:15.819 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50290.log 2023-10-30T12:59:15.820 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2023-10-30T12:59:15.825 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44555.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44555.log.gz 2023-10-30T12:59:15.825 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42970.log 2023-10-30T12:59:15.826 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50454.log 2023-10-30T12:59:15.826 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50290.log.gz 2023-10-30T12:59:15.826 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42970.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2023-10-30T12:59:15.826 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45742.log 2023-10-30T12:59:15.827 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2023-10-30T12:59:15.827 INFO:teuthology.orchestra.run.smithi008.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50454.log.gz 2023-10-30T12:59:15.827 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45742.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45742.log.gz 2023-10-30T12:59:15.827 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50827.log 2023-10-30T12:59:15.828 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2023-10-30T12:59:15.828 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2023-10-30T12:59:15.828 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50827.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50827.log.gz 2023-10-30T12:59:15.829 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50801.log 2023-10-30T12:59:15.829 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47409.log 2023-10-30T12:59:15.829 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2023-10-30T12:59:15.829 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50801.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50801.log.gz 2023-10-30T12:59:15.830 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45608.log 2023-10-30T12:59:15.830 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47409.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47409.log.gz 2023-10-30T12:59:15.830 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44019.log 2023-10-30T12:59:15.831 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45608.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45608.log.gz 2023-10-30T12:59:15.831 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2023-10-30T12:59:15.831 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44019.log.gz 2023-10-30T12:59:15.831 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-30T12:59:15.832 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43541.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2023-10-30T12:59:15.832 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2023-10-30T12:59:15.832 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2023-10-30T12:59:15.833 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49945.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2023-10-30T12:59:15.836 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2023-10-30T12:59:15.836 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51362.log 2023-10-30T12:59:15.836 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2023-10-30T12:59:15.837 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49265.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2023-10-30T12:59:15.837 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51390.log 2023-10-30T12:59:15.837 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51362.log.gz 2023-10-30T12:59:15.837 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44659.log 2023-10-30T12:59:15.838 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51390.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-30T12:59:15.838 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51390.log.gz 2023-10-30T12:59:15.838 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44659.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44659.log.gz 2023-10-30T12:59:15.838 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2023-10-30T12:59:15.839 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50483.log 2023-10-30T12:59:15.839 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2023-10-30T12:59:15.839 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2023-10-30T12:59:15.841 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42728.log 2023-10-30T12:59:15.841 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50483.log.gz 2023-10-30T12:59:15.842 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44763.log 2023-10-30T12:59:15.842 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2023-10-30T12:59:15.842 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42646.log 2023-10-30T12:59:15.842 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42728.log.gz 2023-10-30T12:59:15.843 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48404.log 2023-10-30T12:59:15.843 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44763.log.gz 2023-10-30T12:59:15.844 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2023-10-30T12:59:15.844 INFO:teuthology.orchestra.run.smithi008.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42646.log.gz 2023-10-30T12:59:15.845 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51549.log 2023-10-30T12:59:15.846 INFO:teuthology.orchestra.run.smithi008.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48404.log.gz 2023-10-30T12:59:15.847 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2023-10-30T12:59:15.847 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2023-10-30T12:59:15.848 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49319.log 2023-10-30T12:59:15.849 INFO:teuthology.orchestra.run.smithi008.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51549.log.gz 2023-10-30T12:59:15.849 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51258.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2023-10-30T12:59:15.850 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49810.log 2023-10-30T12:59:15.851 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48167.log 2023-10-30T12:59:15.851 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49319.log.gz 2023-10-30T12:59:15.853 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49810.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49810.log.gz 2023-10-30T12:59:15.853 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44737.log 2023-10-30T12:59:15.854 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2023-10-30T12:59:15.855 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48167.log.gz 2023-10-30T12:59:15.856 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44737.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44737.log.gz 2023-10-30T12:59:15.856 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2023-10-30T12:59:15.857 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48138.log 2023-10-30T12:59:15.858 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2023-10-30T12:59:15.859 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2023-10-30T12:59:15.859 INFO:teuthology.orchestra.run.smithi008.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2023-10-30T12:59:15.860 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48138.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48138.log.gz 2023-10-30T12:59:15.860 INFO:teuthology.orchestra.run.smithi008.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2023-10-30T12:59:15.862 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2023-10-30T12:59:15.863 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz 2023-10-30T12:59:15.863 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42865.log 2023-10-30T12:59:15.864 INFO:teuthology.orchestra.run.smithi008.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2023-10-30T12:59:15.865 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2023-10-30T12:59:15.865 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2023-10-30T12:59:15.867 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43332.log 2023-10-30T12:59:15.867 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42865.log.gz 2023-10-30T12:59:15.868 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48272.log 2023-10-30T12:59:15.868 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2023-10-30T12:59:15.870 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2023-10-30T12:59:15.870 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43332.log.gz 2023-10-30T12:59:15.871 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48272.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48272.log.gz 2023-10-30T12:59:15.871 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49838.log 2023-10-30T12:59:15.873 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51601.log 2023-10-30T12:59:15.873 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2023-10-30T12:59:15.874 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47776.log 2023-10-30T12:59:15.874 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49838.log.gz 2023-10-30T12:59:15.875 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50565.log 2023-10-30T12:59:15.876 INFO:teuthology.orchestra.run.smithi008.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51601.log.gz 2023-10-30T12:59:15.877 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47776.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.47776.log.gz -- 2023-10-30T12:59:15.878 INFO:teuthology.orchestra.run.smithi008.stderr: /var/log/ceph/ceph-client.admin.43993.log 2023-10-30T12:59:15.878 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2023-10-30T12:59:15.879 INFO:teuthology.orchestra.run.smithi008.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50565.log.gz 2023-10-30T12:59:15.880 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2023-10-30T12:59:15.880 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2023-10-30T12:59:15.881 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47323.log 2023-10-30T12:59:15.882 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz 2023-10-30T12:59:15.883 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43048.log 2023-10-30T12:59:15.883 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2023-10-30T12:59:15.885 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2023-10-30T12:59:15.885 INFO:teuthology.orchestra.run.smithi008.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47323.log.gz 2023-10-30T12:59:15.886 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2023-10-30T12:59:15.887 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43048.log.gz 2023-10-30T12:59:15.887 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48950.log 2023-10-30T12:59:15.888 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2023-10-30T12:59:15.889 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51653.log 2023-10-30T12:59:15.889 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2023-10-30T12:59:15.890 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50669.log 2023-10-30T12:59:15.891 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48950.log.gz 2023-10-30T12:59:15.892 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47648.log 2023-10-30T12:59:15.892 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51653.log.gz 2023-10-30T12:59:15.894 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2023-10-30T12:59:15.894 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50669.log.gz 2023-10-30T12:59:15.894 INFO:teuthology.orchestra.run.smithi008.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-30T12:59:15.896 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50160.log 2023-10-30T12:59:15.896 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47648.log.gz 2023-10-30T12:59:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43752.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2023-10-30T12:59:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49893.log 2023-10-30T12:59:15.898 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48246.log 2023-10-30T12:59:15.898 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50160.log.gz 2023-10-30T12:59:15.899 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49893.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49893.log.gz 2023-10-30T12:59:15.899 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42761.log 2023-10-30T12:59:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43327.log 2023-10-30T12:59:15.901 INFO:teuthology.orchestra.run.smithi008.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48246.log.gz 2023-10-30T12:59:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42761.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42761.log.gz 2023-10-30T12:59:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48976.log 2023-10-30T12:59:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43327.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-30T12:59:15.904 INFO:teuthology.orchestra.run.smithi008.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43327.log.gz 2023-10-30T12:59:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48976.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48976.log.gz 2023-10-30T12:59:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44685.log 2023-10-30T12:59:15.907 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33182.log 2023-10-30T12:59:15.908 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2023-10-30T12:59:15.909 INFO:teuthology.orchestra.run.smithi008.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44685.log.gz 2023-10-30T12:59:15.909 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph.tmp-client.admin.33182.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33182.log.gz 2023-10-30T12:59:15.910 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44841.log 2023-10-30T12:59:15.911 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51232.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2023-10-30T12:59:15.912 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47921.log 2023-10-30T12:59:15.913 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49186.log 2023-10-30T12:59:15.913 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44841.log.gz 2023-10-30T12:59:15.914 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47921.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.47921.log.gz -5 2023-10-30T12:59:15.915 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2023-10-30T12:59:15.916 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50643.log 2023-10-30T12:59:15.916 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49186.log.gz 2023-10-30T12:59:15.917 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49919.log 2023-10-30T12:59:15.917 INFO:teuthology.orchestra.run.smithi008.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2023-10-30T12:59:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50643.log.gz 2023-10-30T12:59:15.919 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45282.log 2023-10-30T12:59:15.921 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50368.log 2023-10-30T12:59:15.921 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49919.log.gz 2023-10-30T12:59:15.922 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2023-10-30T12:59:15.922 INFO:teuthology.orchestra.run.smithi008.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45282.log.gz 2023-10-30T12:59:15.924 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48194.log 2023-10-30T12:59:15.924 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50368.log.gz 2023-10-30T12:59:15.925 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49725.log 2023-10-30T12:59:15.925 INFO:teuthology.orchestra.run.smithi008.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2023-10-30T12:59:15.926 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48194.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48194.log.gz 2023-10-30T12:59:15.927 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2023-10-30T12:59:15.928 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49725.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-30T12:59:15.929 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49725.log.gz 2023-10-30T12:59:15.930 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 87.9% 2023-10-30T12:59:15.930 INFO:teuthology.orchestra.run.smithi008.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2023-10-30T12:59:15.931 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2023-10-30T12:59:15.933 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2023-10-30T12:59:15.933 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2023-10-30T12:59:15.934 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51024.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2023-10-30T12:59:15.935 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43837.log 2023-10-30T12:59:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50617.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2023-10-30T12:59:15.936 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2023-10-30T12:59:15.938 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50316.log 2023-10-30T12:59:15.938 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43837.log.gz 2023-10-30T12:59:15.939 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50125.log 2023-10-30T12:59:15.939 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2023-10-30T12:59:15.941 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50428.log 2023-10-30T12:59:15.941 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2023-10-30T12:59:15.942 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50511.log 2023-10-30T12:59:15.942 INFO:teuthology.orchestra.run.smithi008.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50125.log.gz 2023-10-30T12:59:15.944 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2023-10-30T12:59:15.944 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50428.log.gz 2023-10-30T12:59:15.945 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50511.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50511.log.gz 2023-10-30T12:59:15.945 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2023-10-30T12:59:15.947 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43543.log 2023-10-30T12:59:15.947 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2023-10-30T12:59:15.948 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43889.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2023-10-30T12:59:15.948 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42996.log 2023-10-30T12:59:15.950 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49867.log 2023-10-30T12:59:15.950 INFO:teuthology.orchestra.run.smithi008.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43543.log.gz 2023-10-30T12:59:15.951 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2023-10-30T12:59:15.952 INFO:teuthology.orchestra.run.smithi008.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42996.log.gz 2023-10-30T12:59:15.953 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49867.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49867.log.gz 2023-10-30T12:59:15.953 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48300.log 2023-10-30T12:59:15.954 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43321.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2023-10-30T12:59:15.955 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49291.log 2023-10-30T12:59:15.956 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42839.log 2023-10-30T12:59:15.957 INFO:teuthology.orchestra.run.smithi008.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48300.log.gz 2023-10-30T12:59:15.958 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2023-10-30T12:59:15.958 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49291.log.gz 2023-10-30T12:59:15.959 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2023-10-30T12:59:15.960 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49239.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2023-10-30T12:59:15.961 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47490.log 2023-10-30T12:59:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43532.log 2023-10-30T12:59:15.964 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43967.log 2023-10-30T12:59:15.965 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47490.log.gz 2023-10-30T12:59:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2023-10-30T12:59:15.966 INFO:teuthology.orchestra.run.smithi008.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gz 2023-10-30T12:59:15.967 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43967.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43967.log.gz -5 2023-10-30T12:59:15.968 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51627.log 2023-10-30T12:59:15.969 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48001.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T12:59:15.969 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2023-10-30T12:59:15.970 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51627.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.51627.log.gz 2023-10-30T12:59:15.970 INFO:teuthology.orchestra.run.smithi008.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45530.log 2023-10-30T12:59:15.972 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48326.log 2023-10-30T12:59:15.973 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2023-10-30T12:59:15.974 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48326.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-30T12:59:15.975 INFO:teuthology.orchestra.run.smithi008.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48326.log.gz 2023-10-30T12:59:15.975 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2023-10-30T12:59:15.977 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49213.log 2023-10-30T12:59:15.978 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43335.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2023-10-30T12:59:15.980 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49213.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49213.log.gz 2023-10-30T12:59:15.981 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2023-10-30T12:59:15.982 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 2023-10-30T12:59:15.984 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz -- 2023-10-30T12:59:15.984 INFO:teuthology.orchestra.run.smithi008.stderr: /var/log/ceph/ceph-client.admin.50238.log 2023-10-30T12:59:15.985 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49108.log 2023-10-30T12:59:15.985 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2023-10-30T12:59:15.986 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50238.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.50238.log.gz 2023-10-30T12:59:15.987 INFO:teuthology.orchestra.run.smithi008.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2023-10-30T12:59:15.988 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49108.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49108.log.gz 2023-10-30T12:59:15.988 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2023-10-30T12:59:15.990 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2023-10-30T12:59:15.990 INFO:teuthology.orchestra.run.smithi008.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gz 2023-10-30T12:59:15.991 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43785.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2023-10-30T12:59:15.992 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44451.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2023-10-30T12:59:15.993 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2023-10-30T12:59:15.994 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47542.log 2023-10-30T12:59:15.996 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49751.log 2023-10-30T12:59:15.996 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2023-10-30T12:59:15.997 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47542.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47542.log.gz 2023-10-30T12:59:15.999 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49751.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49751.log.gz 2023-10-30T12:59:15.999 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2023-10-30T12:59:16.000 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2023-10-30T12:59:16.001 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-30T12:59:16.002 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.50745.log 2023-10-30T12:59:16.002 INFO:teuthology.orchestra.run.smithi008.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2023-10-30T12:59:16.004 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.38271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47674.log 2023-10-30T12:59:16.004 INFO:teuthology.orchestra.run.smithi008.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2023-10-30T12:59:16.005 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50745.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2023-10-30T12:59:16.006 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47674.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47674.log.gz 2023-10-30T12:59:16.007 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44045.log 2023-10-30T12:59:16.008 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2023-10-30T12:59:16.010 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2023-10-30T12:59:16.011 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44045.log.gz 2023-10-30T12:59:16.012 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47217.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gzgzip 2023-10-30T12:59:16.012 INFO:teuthology.orchestra.run.smithi008.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2023-10-30T12:59:16.013 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44306.log 2023-10-30T12:59:16.014 INFO:teuthology.orchestra.run.smithi008.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2023-10-30T12:59:16.015 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2023-10-30T12:59:16.015 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45364.log 2023-10-30T12:59:16.016 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45228.log 2023-10-30T12:59:16.017 INFO:teuthology.orchestra.run.smithi008.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44306.log.gz 2023-10-30T12:59:16.018 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45364.log.gz 2023-10-30T12:59:16.019 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45228.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45228.log.gz 2023-10-30T12:59:16.020 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45582.log 2023-10-30T12:59:16.021 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49423.log 2023-10-30T12:59:16.023 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44251.log 2023-10-30T12:59:16.023 INFO:teuthology.orchestra.run.smithi008.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45582.log.gz 2023-10-30T12:59:16.025 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44503.log 2023-10-30T12:59:16.025 INFO:teuthology.orchestra.run.smithi008.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49423.log.gz 2023-10-30T12:59:16.026 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51679.log 2023-10-30T12:59:16.027 INFO:teuthology.orchestra.run.smithi008.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2023-10-30T12:59:16.028 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44503.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44503.log.gz 2023-10-30T12:59:16.029 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51679.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51679.log.gz 2023-10-30T12:59:16.030 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51575.log 2023-10-30T12:59:16.031 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2023-10-30T12:59:16.033 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51575.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51575.log.gz 2023-10-30T12:59:16.033 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44071.log 2023-10-30T12:59:16.034 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2023-10-30T12:59:16.034 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2023-10-30T12:59:16.035 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44071.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44071.log.gz 2023-10-30T12:59:16.037 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45397.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2023-10-30T12:59:16.037 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2023-10-30T12:59:16.039 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T12:59:16.041 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43538.log 2023-10-30T12:59:16.042 INFO:teuthology.orchestra.run.smithi008.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2023-10-30T12:59:16.043 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47297.log 2023-10-30T12:59:16.044 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43538.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T12:59:16.044 INFO:teuthology.orchestra.run.smithi008.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43538.log.gz 2023-10-30T12:59:16.045 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47297.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2023-10-30T12:59:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-30T12:59:16.047 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-30T12:59:16.049 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2023-10-30T12:59:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.43941.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2023-10-30T12:59:16.052 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2023-10-30T12:59:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50998.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2023-10-30T12:59:16.056 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38152.log 2023-10-30T12:59:16.059 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.38152.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38152.log.gz 2023-10-30T12:59:16.059 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2023-10-30T12:59:16.062 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42537.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2023-10-30T12:59:16.063 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44815.log 2023-10-30T12:59:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44815.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44815.log.gz 2023-10-30T12:59:16.066 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2023-10-30T12:59:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2023-10-30T12:59:16.069 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48352.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2023-10-30T12:59:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44477.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2023-10-30T12:59:16.073 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43811.log 2023-10-30T12:59:16.076 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43811.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43811.log.gz 2023-10-30T12:59:16.076 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49002.log 2023-10-30T12:59:16.079 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49002.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49002.log.gz 2023-10-30T12:59:16.080 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50212.log 2023-10-30T12:59:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50212.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50212.log.gz 2023-10-30T12:59:16.083 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2023-10-30T12:59:16.086 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47817.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2023-10-30T12:59:16.087 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2023-10-30T12:59:16.090 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47895.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gz 2023-10-30T12:59:16.090 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43315.log 2023-10-30T12:59:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43315.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43315.log.gz 2023-10-30T12:59:16.093 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2023-10-30T12:59:16.096 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42392.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2023-10-30T12:59:16.097 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51769.log 2023-10-30T12:59:16.099 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51769.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51769.log.gz 2023-10-30T12:59:16.100 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2023-10-30T12:59:16.103 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49452.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2023-10-30T12:59:16.104 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42787.log 2023-10-30T12:59:16.107 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42787.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42787.log.gz 2023-10-30T12:59:16.107 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2023-10-30T12:59:16.110 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51076.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2023-10-30T12:59:16.110 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2023-10-30T12:59:16.113 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2023-10-30T12:59:16.114 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51442.log 2023-10-30T12:59:16.116 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2023-10-30T12:59:16.118 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51442.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51442.log.gz 2023-10-30T12:59:16.120 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49028.log 2023-10-30T12:59:16.121 INFO:teuthology.orchestra.run.smithi008.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2023-10-30T12:59:16.124 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45178.log 2023-10-30T12:59:16.125 INFO:teuthology.orchestra.run.smithi008.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49028.log.gz 2023-10-30T12:59:16.128 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45178.log.gz 2023-10-30T12:59:16.128 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50591.log 2023-10-30T12:59:16.132 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47843.log 2023-10-30T12:59:16.132 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50591.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50591.log.gz 2023-10-30T12:59:16.133 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-30T12:59:16.135 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47843.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47843.log.gz 2023-10-30T12:59:16.135 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51416.log 2023-10-30T12:59:16.138 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51416.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51416.log.gz 2023-10-30T12:59:16.139 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45338.log 2023-10-30T12:59:16.142 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45338.log.gz 2023-10-30T12:59:16.142 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50879.log 2023-10-30T12:59:16.145 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50879.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50879.log.gz 2023-10-30T12:59:16.146 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42891.log 2023-10-30T12:59:16.148 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42891.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42891.log.gz 2023-10-30T12:59:16.149 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43127.log 2023-10-30T12:59:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43127.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43127.log.gz 2023-10-30T12:59:16.153 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42594.log 2023-10-30T12:59:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42594.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42594.log.gz 2023-10-30T12:59:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2023-10-30T12:59:16.159 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50186.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2023-10-30T12:59:16.160 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45634.log 2023-10-30T12:59:16.163 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45634.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45634.log.gz 2023-10-30T12:59:16.163 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45423.log 2023-10-30T12:59:16.167 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45423.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45423.log.gz 2023-10-30T12:59:16.167 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43109.log 2023-10-30T12:59:16.170 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43109.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43109.log.gz 2023-10-30T12:59:16.171 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49160.log 2023-10-30T12:59:16.174 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49160.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49160.log.gz 2023-10-30T12:59:16.174 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44633.log 2023-10-30T12:59:16.178 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44633.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44633.log.gz 2023-10-30T12:59:16.178 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2023-10-30T12:59:16.181 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2023-10-30T12:59:16.181 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50342.log 2023-10-30T12:59:16.184 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50342.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50342.log.gz 2023-10-30T12:59:16.185 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2023-10-30T12:59:16.187 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43334.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2023-10-30T12:59:16.188 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44429.log 2023-10-30T12:59:16.190 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.0.44429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44429.log.gz 2023-10-30T12:59:16.191 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2023-10-30T12:59:16.194 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44581.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2023-10-30T12:59:16.195 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44529.log 2023-10-30T12:59:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44529.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44529.log.gz 2023-10-30T12:59:16.199 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2023-10-30T12:59:16.201 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47949.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2023-10-30T12:59:16.202 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45256.log 2023-10-30T12:59:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45256.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45256.log.gz 2023-10-30T12:59:16.205 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2023-10-30T12:59:16.209 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48053.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz 2023-10-30T12:59:16.209 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43520.log 2023-10-30T12:59:16.212 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43520.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43520.log.gz 2023-10-30T12:59:16.213 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50775.log 2023-10-30T12:59:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50775.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50775.log.gz 2023-10-30T12:59:16.216 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45312.log 2023-10-30T12:59:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.45312.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45312.log.gz 2023-10-30T12:59:16.219 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2023-10-30T12:59:16.222 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2023-10-30T12:59:16.223 INFO:teuthology.orchestra.run.smithi008.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2023-10-30T12:59:16.225 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51525.log 2023-10-30T12:59:16.227 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43863.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2023-10-30T12:59:16.228 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51525.log: -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2023-10-30T12:59:16.228 INFO:teuthology.orchestra.run.smithi008.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51525.log.gz 2023-10-30T12:59:16.230 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2023-10-30T12:59:16.231 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2023-10-30T12:59:16.238 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2023-10-30T12:59:16.243 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2023-10-30T12:59:16.244 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51128.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2023-10-30T12:59:16.248 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50049.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2023-10-30T12:59:16.249 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44607.log 2023-10-30T12:59:16.254 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50853.log 2023-10-30T12:59:16.256 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49971.log 2023-10-30T12:59:16.256 INFO:teuthology.orchestra.run.smithi008.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44607.log.gz 2023-10-30T12:59:16.263 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.50853.log: /var/log/ceph/ceph-client.admin.49971.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50853.log.gz 2023-10-30T12:59:16.263 INFO:teuthology.orchestra.run.smithi008.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2023-10-30T12:59:16.264 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2023-10-30T12:59:16.266 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47375.log 2023-10-30T12:59:16.268 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42428.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz -5 2023-10-30T12:59:16.268 INFO:teuthology.orchestra.run.smithi008.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51469.log 2023-10-30T12:59:16.270 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48220.log 2023-10-30T12:59:16.271 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.51469.log: /var/log/ceph/ceph-client.admin.47375.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51469.log.gz 2023-10-30T12:59:16.271 INFO:teuthology.orchestra.run.smithi008.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47375.log.gz 2023-10-30T12:59:16.274 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42568.log 2023-10-30T12:59:16.276 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48220.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48220.log.gz 2023-10-30T12:59:16.279 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42568.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42568.log.gz 2023-10-30T12:59:16.281 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2023-10-30T12:59:16.286 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48086.log 2023-10-30T12:59:16.288 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47705.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2023-10-30T12:59:16.291 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47516.log 2023-10-30T12:59:16.292 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.48086.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48086.log.gz 2023-10-30T12:59:16.294 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42672.log 2023-10-30T12:59:16.296 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.47516.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47516.log.gz 2023-10-30T12:59:16.298 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2023-10-30T12:59:16.300 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42672.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42672.log.gz 2023-10-30T12:59:16.301 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.42324.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz 2023-10-30T12:59:16.302 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2023-10-30T12:59:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.49134.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2023-10-30T12:59:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2023-10-30T12:59:16.314 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43074.log 2023-10-30T12:59:16.316 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43336.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2023-10-30T12:59:16.318 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2023-10-30T12:59:16.320 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43074.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43074.log.gz 2023-10-30T12:59:16.322 INFO:teuthology.orchestra.run.smithi008.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44150.log 2023-10-30T12:59:16.324 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.43323.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2023-10-30T12:59:16.332 INFO:teuthology.orchestra.run.smithi008.stderr:/var/log/ceph/ceph-client.admin.44150.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44150.log.gz 2023-10-30T12:59:16.681 INFO:teuthology.orchestra.run.smithi008.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-30T12:59:17.961 INFO:teuthology.orchestra.run.smithi046.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-30T12:59:17.994 INFO:teuthology.orchestra.run.smithi008.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-30T12:59:18.272 INFO:teuthology.orchestra.run.smithi008.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-30T12:59:18.610 INFO:teuthology.orchestra.run.smithi046.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-30T12:59:18.842 INFO:teuthology.orchestra.run.smithi046.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-30T12:59:19.202 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-30T12:59:19.370 INFO:teuthology.orchestra.run.smithi008.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-30T12:59:19.801 INFO:teuthology.orchestra.run.smithi008.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-30T12:59:23.129 INFO:teuthology.orchestra.run.smithi008.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-30T12:59:26.267 INFO:teuthology.orchestra.run.smithi008.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-30T12:59:26.427 INFO:teuthology.orchestra.run.smithi008.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2023-10-30T12:59:26.429 INFO:teuthology.orchestra.run.smithi008.stderr: 2023-10-30T12:59:26.429 INFO:teuthology.orchestra.run.smithi008.stderr:real 0m10.689s 2023-10-30T12:59:26.429 INFO:teuthology.orchestra.run.smithi008.stderr:user 0m37.942s 2023-10-30T12:59:26.430 INFO:teuthology.orchestra.run.smithi008.stderr:sys 0m3.313s 2023-10-30T12:59:28.687 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2023-10-30T12:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-10-30T12:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m12.925s 2023-10-30T12:59:28.688 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m24.199s 2023-10-30T12:59:28.689 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.810s 2023-10-30T12:59:28.690 INFO:tasks.ceph:Archiving logs... 2023-10-30T12:59:28.690 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/log/ceph to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi008/log 2023-10-30T12:59:28.692 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T12:59:32.487 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi046/log 2023-10-30T12:59:32.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T12:59:35.061 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-30T12:59:35.071 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-30T12:59:35.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T12:59:35.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T12:59:35.147 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-30T12:59:35.156 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-30T12:59:35.157 DEBUG:teuthology.orchestra.run.smithi008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:59:35.161 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T12:59:35.174 INFO:teuthology.orchestra.run.smithi008.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:59:35.175 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================== 2023-10-30T12:59:35.175 INFO:teuthology.orchestra.run.smithi008.stdout:*hv01.front.sepi 44.4.53.2 2 u 42 64 377 0.086 -2.171 0.503 2023-10-30T12:59:35.175 INFO:teuthology.orchestra.run.smithi008.stdout:+hv02.front.sepi 63.231.80.2 3 u 42 64 377 0.107 +0.863 0.469 2023-10-30T12:59:35.176 INFO:teuthology.orchestra.run.smithi008.stdout:+hv03.front.sepi 74.6.168.72 3 u 51 64 377 0.108 -0.621 0.458 2023-10-30T12:59:35.176 INFO:teuthology.orchestra.run.smithi008.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-10-30T12:59:35.177 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-10-30T12:59:35.178 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-10-30T12:59:35.178 INFO:teuthology.orchestra.run.smithi046.stdout:*hv01.front.sepi 44.4.53.2 2 u 53 64 377 0.084 -3.125 0.205 2023-10-30T12:59:35.178 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 43 64 377 0.080 +0.758 0.151 2023-10-30T12:59:35.179 INFO:teuthology.orchestra.run.smithi046.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.096 -1.531 0.201 2023-10-30T12:59:35.179 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-10-30T12:59:35.179 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-30T12:59:35.189 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-30T12:59:35.189 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-30T12:59:35.198 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-30T12:59:35.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-30T12:59:35.216 INFO:teuthology.task.internal:Duration was 1629.129862 seconds 2023-10-30T12:59:35.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-30T12:59:35.226 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-30T12:59:35.226 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T12:59:35.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T12:59:35.315 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-30T12:59:35.316 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi008.front.sepia.ceph.com 2023-10-30T12:59:35.316 DEBUG:teuthology.orchestra.run.smithi008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T12:59:35.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2023-10-30T12:59:35.340 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T12:59:35.350 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-30T12:59:35.350 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T12:59:35.384 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T12:59:35.419 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-30T12:59:35.419 DEBUG:teuthology.orchestra.run.smithi008:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T12:59:35.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T12:59:35.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-30T12:59:35.549 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-30T12:59:35.549 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T12:59:35.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T12:59:35.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-30T12:59:35.596 DEBUG:teuthology.orchestra.run.smithi008:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:59:35.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:59:35.632 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = core 2023-10-30T12:59:35.633 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2023-10-30T12:59:35.655 DEBUG:teuthology.orchestra.run.smithi008:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:59:35.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:59:35.701 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T12:59:35.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T12:59:35.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-30T12:59:35.720 INFO:teuthology.task.internal:Transferring archived files... 2023-10-30T12:59:35.723 DEBUG:teuthology.misc:Transferring archived files from smithi008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi008 2023-10-30T12:59:35.724 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T12:59:35.771 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032/remote/smithi046 2023-10-30T12:59:35.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T12:59:35.793 INFO:teuthology.task.internal:Removing archive directory... 2023-10-30T12:59:35.793 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T12:59:35.812 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T12:59:35.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-30T12:59:35.851 INFO:teuthology.task.internal:Not uploading archives. 2023-10-30T12:59:35.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-30T12:59:35.861 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-30T12:59:35.861 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T12:59:35.868 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T12:59:35.874 INFO:teuthology.orchestra.run.smithi008.stdout: 657972 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 12:59 /home/ubuntu/cephtest 2023-10-30T12:59:35.892 INFO:teuthology.orchestra.run.smithi046.stdout: 524292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 12:59 /home/ubuntu/cephtest 2023-10-30T12:59:35.893 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-30T12:59:35.904 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-30T12:59:35.926 INFO:teuthology.nuke:Checking targets against current locks 2023-10-30T12:59:35.952 DEBUG:teuthology.nuke:shortname: smithi008 2023-10-30T12:59:35.952 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:59:35.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:23:14.879626', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiIcfgwrGC9+v3RXe8vNJeksoOyZsreQ1gqopuMNh7A6b/pQ2xJyT5FrS1lYTxOOiBa46uoctB44j3nvKR85mw='} 2023-10-30T12:59:35.979 DEBUG:teuthology.nuke:shortname: smithi046 2023-10-30T12:59:35.979 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T12:59:36.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-10-30_06:34:16-fs-wip-lusov-balance-after-stop-distro-default-smithi/7441032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 12:23:14.881088', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrOaSl/1vsdItzZTYKMwfRP6iubhYHTiH89lRxHjBRZoyL5v6NbgZ2wXiui+zGvgcvVlPNuVTJ+jWYb+TFCaoA='} 2023-10-30T12:59:36.030 INFO:teuthology.orchestra.console.smithi008:Power off 2023-10-30T12:59:36.031 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T12:59:36.055 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2023-10-30T12:59:36.055 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:36.072 INFO:teuthology.orchestra.console.smithi046:Power off 2023-10-30T12:59:36.073 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T12:59:36.083 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-10-30T12:59:36.097 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-10-30T12:59:36.097 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:36.134 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-30T12:59:40.086 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:40.136 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:40.197 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-10-30T12:59:40.247 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-30T12:59:44.202 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:44.248 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:44.312 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2023-10-30T12:59:44.358 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-30T12:59:44.414 INFO:teuthology.orchestra.console.smithi008:Power off completed 2023-10-30T12:59:48.362 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T12:59:48.472 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-30T12:59:48.573 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-10-30T12:59:48.680 INFO:teuthology.run:Summary data: client.0-kernel-sha1: eacf00cc51e204f401b4d5d4f84825a68f373c74 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 1629.1298620700836 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi008 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=00c1c13cea21a686b046584596fbeac5315a7d71 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e9109f81da8f44398e6a4e79673d840d status: fail success: false 2023-10-30T12:59:48.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T12:59:48.780 INFO:teuthology.run:FAIL